Guide-yakabatana mibvunzo · Pakati Nhanho

Multi-Query Attention Quiz

Edza kuti Multi-Query Attention inogovera sei-kiyi-kukosha misoro, inoderedza cache traffic, uye ine chekuita neakaunganidzwa-mubvunzo kutarisisa mumhando dzedecoder.

Nzira dzekutungamira dzinoenderanaMulti Query Attention
Mubvunzo 1 ye 8

Chii chinogovera Multi-Query Attention pamisoro yese yekutarisisa?

Tarisa mubvunzo wese8 mibvunzo

Mhinduro uye tsananguro zvinoonekwa paunenge uchitora bvunzo.

  1. Chii chinogovera Multi-Query Attention pamisoro yese yekutarisisa?
    • Seti imwe chete yemakiyi uye kukosha
    • Maonero emubvunzo
    • The softmax tembiricha
    • The output layer uremu
  2. Ndeipi yekutanga bhodhoro iyo MQA inogadzirisa panguva yekugadzira autoregressive?
    • Floating-point compute throughput
    • Memory bandwidth kubva pakurodha iyo KV cache
    • Huwandu hwenguva dzekudzidziswa
    • Tokenizer saizi yemazwi
  3. Nekuda kwechinhu chipi chinoita kuti MQA idzore cache yeKV?
    • Nekutevedzana kureba
    • Nekukura kwebatch
    • Nenhamba yekutarisisa misoro
    • Izvo hazvichinje saizi yeKV cache
  4. Ndeipi kutengeserana kukuru kwekushandisa MQA?
    • Kunyanya kunonoka kufungidzira
    • Haichakwanisi kushandisa misoro yakawanda zvachose
    • Zvinoda kure kure kudzidziswa data
    • Kudonha zvine mwero mumhando yemhando
  5. Ndeupi musiyano unogara pakati peyakajairwa-yepamusoro-soro kutarisa uye MQA?
    • Grouped-Query Attention
    • Sliding-Window Attention
    • Flash Attention (Kutarisa Kwemheni)
    • Sparse Attention
  6. Ndiani akakurudzira Multi-Query Attention?
    • Geoffrey Hinton
    • Noam Shazeer
    • Yann LeCun
    • Andrej Karpathy
  7. In Multi-Query Attention, what remains separate for each attention head?
    • The shared key projection
    • The shared value projection
    • The query projection
    • The token sequence itself
  8. How does Grouped-Query Attention differ from standard MQA?
    • It shares query projections but gives every head separate keys and values
    • It removes attention and replaces it with a recurrent layer
    • It groups output vocabulary tokens instead of attention heads
    • It shares keys and values within several groups of query heads rather than across all heads at once