Guide-yakabatana mibvunzo · Pakati Nhanho

Next-Token Prediction Quiz

Review autoregressive language-model training, causal context, teacher forcing and the difference between training and generation.

Nzira dzekutungamira dzinoenderanaNext Token Prediction
Mubvunzo 1 ye 8

Chii chinotevera-chiratidzo chekufungidzira modhi inobuda padanho rega rega?

Tarisa mubvunzo wese8 mibvunzo

Mhinduro uye tsananguro zvinoonekwa paunenge uchitora bvunzo.

  1. Chii chinotevera-chiratidzo chekufungidzira modhi inobuda padanho rega rega?
    • Mukana wekugovera pamusoro pemazwi echiratidzo chinotevera
    • Izwi rimwechete rakagadziriswa
    • Kushandurwa kwechirevo
    • Chibodzwa chegirama
  2. Imhando ipi yekutarisisa inoshandiswa neGPT-style decoder?
    • Bidirectional attention
    • Causal (masked) kutarisa kunovanza zviratidzo zveramangwana
    • Hapana kana hanya zvachose
    • Random attention
  3. Chii chinonzi 'autoregressive generation' chinorevei pano?
    • Iyo modhi inogadzira tokeni dzese kamwechete
    • Iyo modhi inodzokera kune yakakosha kukosha
    • Imwe neimwe tokeni nyowani inowedzerwa uye inodzoserwa mukati kufanotaura chinotevera
    • Tokens inogadzirwa zvisina kurongeka
  4. Ndeipi basa rekurasikirwa rinowanzo kuderedzwa panguva yekudzidziswa?
    • Kureva kukanganisa kwakapetwa kaviri
    • Hinge kurasikirwa
    • Cosine chinhambwe
    • Yambuka-entropy pamusoro pechiratidzo chinotevera
  5. Chii chinoita kuti kukwidza tembiricha panguva yesampling?
    • Inoita kuti zvinobuda zvive zvisina tsarukano uye kugadzira
    • Inoita inobuda yakanyatso deterministic
    • Inowedzera saizi yemazwi
    • Inobvisa softmax layer
  6. Pamberi pekufanotaura, sei mavara asina kunyorwa anowanzo gadzirwa?
    • Rinoturikirwa kuChirungu
    • Iyo inokamurwa kuita tokeni nechiratidzo
    • Inoshandurwa kuita audio
    • Inosiiwa semavara mambishi pasina kugadziridzwa
  7. During next-token training, what is teacher forcing?
    • The model receives future target tokens before the input
    • The model randomly changes the tokenizer at every step
    • The model receives the ground-truth preceding tokens when predicting the next token
    • The model always samples its own previous prediction during training
  8. What mismatch can arise because generation feeds back model-produced tokens?
    • The model can no longer use attention
    • The vocabulary disappears after the first token
    • The model’s loss is computed only on images
    • Errors in generated prefixes can affect later predictions, unlike training on the ground-truth prefix