Guide-yakabatana mibvunzo · Zvakaoma Nhanho

Attention Mechanisms Quiz

Yedza query-kiyi-kukosha kutarisisa, softmax kuyera, misoro yakafanana, dense yekutarisisa mutengo, huremu-kukosha kuverenga, uye causal masking yechizvarwa.

Nzira dzekutungamira dzinoenderanaMekanizimu yeKutarisa
Mubvunzo 1 ye 8

What does token attention combine when forming an output representation?

Tarisa mubvunzo wese8 mibvunzo

Mhinduro uye tsananguro zvinoonekwa paunenge uchitora bvunzo.

  1. What does token attention combine when forming an output representation?
    • Permanent copies of the entire conversation on disk
    • Value information from allowed positions using computed weights
    • Only manually chosen keywords with no learned scores
    • A translation dictionary that removes every other token
  2. Ndeipi yemhando yepamusoro 2017 bepa yakaunza iyo transformer architecture?
    • 'Zvakadzika Residual Dzidzo yeKuzivikanwa kweMufananidzo'
    • 'Kutamba Atari Nekudzidza Kwakadzika Kwekusimbisa'
    • 'Kungwarira Ndiko Chete Chaunoda'
    • 'Generative Adversarial Networks'
  3. Ndeapi mavekita matatu akaverengerwa kune imwe neimwe tokeni mukutarisisa?
    • Mubvunzo, kiyi, uye kukosha
    • Input, output, uye bias
    • Mean, median, uye modhi
    • Encoder, decoder, uye kurasikirwa
  4. Muformula softmax(QK^T / √d_k) V, sei kugovanisa ne √d_k?
    • Kushandura mhedzisiro kuita manhamba
    • Kubvisa nhamba dzisina kunaka
    • Kuita kuti modhi imhanye paGPU
    • Kuchengeta zvibodzwa zvedot-chigadzirwa kubva pakunyanya kukura pamberi peSoftmax
  5. Why does dense self-attention involve growing pairwise work for very long token sequences?
    • Nekuti inofanirwa kudhawunirodha data nyowani kubva painternet
    • Nekuti manyoro marefu anogara aine mazwi asingawanzo
    • Nekuti mutengo unokura neskweya yekutevedzana kureba (O(n²))
    • Nekuti inochinjira kune inononoka programming mutauro
  6. What capacity does multi-head attention provide through separate learned projections?
    • A guarantee that every head has one unique named meaning
    • Several attention representations that can capture different relationships in parallel
    • Removal of every token embedding
    • A reduction of vocabulary size by definition
  7. In a scalar attention example, values two and six receive weights three-quarters and one-quarter. What is their weighted output?
    • Zvisere
    • Zvina
    • Vaviri
    • Vatatu
  8. What does a causal attention mask prevent in an autoregressive text decoder?
    • Using later token positions when forming the current position's representation
    • Using any previous token
    • Computing any query or key vectors
    • Learning all model parameters