Idanwo ti o sopọ mọ itọsọna · Lile Ipele

Ifojusi Mechanisms adanwo

Test query-key-value attention, softmax scaling, parallel heads, dense attention cost, weighted-value calculations, and causal masking for generation.

Jẹmọ itọsọna awọn ọnaIfojusi Mechanisms
Ibeere 1 ti 8

What does token attention combine when forming an output representation?

Awotẹlẹ gbogbo ibeere8 ibeere

Awọn idahun ati awọn alaye han bi o ṣe n mu ibeere naa.

  1. What does token attention combine when forming an output representation?
    • Permanent copies of the entire conversation on disk
    • Value information from allowed positions using computed weights
    • Only manually chosen keywords with no learned scores
    • A translation dictionary that removes every other token
  2. Eyi ti enikeji 2017 iwe ṣe awọn transformer faaji?
    • 'Ẹkọ ti o ku jinna fun idanimọ Aworan'
    • 'Ṣiṣere Atari pẹlu Ẹkọ Imudara Jin'
    • 'Akikanju Ni Gbogbo Ohun Ti O Nilo'
    • 'Awọn nẹtiwọki Adversarial Generative'
  3. Kini awọn oniṣiro mẹta ti a ṣe iṣiro fun ami kọọkan ni akiyesi?
    • Ibeere, bọtini, ati iye
    • Iṣagbewọle, iṣejade, ati abosi
    • Itumọ, agbedemeji, ati ipo
    • Encoder, decoder, ati adanu
  4. Ninu agbekalẹ softmax (QK^T / √d_k) V, kilode ti o pin nipasẹ √d_k?
    • Lati yi abajade pada si odidi
    • Lati yọ awọn nọmba odi kuro
    • Lati jẹ ki awoṣe ṣiṣẹ lori GPU kan
    • Lati tọju awọn ikun-ọja aami lati di tobi ju softmax
  5. Why does dense self-attention involve growing pairwise work for very long token sequences?
    • Nitoripe o gbọdọ ṣe igbasilẹ data tuntun lati intanẹẹti
    • Nitori ọrọ gigun nigbagbogbo ni awọn ọrọ ti o ṣọwọn ninu
    • Nitori iye owo dagba pẹlu onigun mẹrin ti gigun ọkọọkan (O(n²))
    • Nitoripe o yipada si ede siseto ti o lọra
  6. What capacity does multi-head attention provide through separate learned projections?
    • A guarantee that every head has one unique named meaning
    • Several attention representations that can capture different relationships in parallel
    • Removal of every token embedding
    • A reduction of vocabulary size by definition
  7. In a scalar attention example, values two and six receive weights three-quarters and one-quarter. What is their weighted output?
    • Mẹjọ
    • Mẹrin
    • Meji
    • Mẹta
  8. What does a causal attention mask prevent in an autoregressive text decoder?
    • Using later token positions when forming the current position's representation
    • Using any previous token
    • Computing any query or key vectors
    • Learning all model parameters