Idanwo ti o sopọ mọ itọsọna · Alabọde Ipele

Olona-Ibeere akiyesi adanwo

Test how Multi-Query Attention shares key-value heads, reduces cache traffic, and relates to grouped-query attention in decoder models.

Jẹmọ itọsọna awọn ọnaOlona Ibere ​​Ifojusi
Ibeere 1 ti 8

Kini Ifarabalẹ Olona-Ibeere pin kọja gbogbo awọn ori akiyesi?

Awotẹlẹ gbogbo ibeere8 ibeere

Awọn idahun ati awọn alaye han bi o ṣe n mu ibeere naa.

  1. Kini Ifarabalẹ Olona-Ibeere pin kọja gbogbo awọn ori akiyesi?
    • Eto kan ti awọn bọtini ati iye
    • Awọn asọtẹlẹ ibeere
    • Awọn iwọn otutu softmax
    • Awọn o wu Layer òṣuwọn
  2. Kini igo akọkọ ti MQA n koju lakoko iran autoregressive?
    • Lilefoofo-ojuami iṣiro losi
    • Bandiwidi iranti lati ikojọpọ kaṣe KV
    • Awọn nọmba ti ikẹkọ epochs
    • Tokenizer iwọn fokabulari
  3. Nipa aijọju kini ifosiwewe MQA ṣe dinku kaṣe KV naa?
    • Nipa ipari ọkọọkan
    • Nipa iwọn ipele
    • Nipa awọn nọmba ti akiyesi olori
    • Ko yi iwọn kaṣe KV pada
  4. Kini iṣowo-pipa akọkọ ti lilo MQA?
    • Elo losokepupo itọkasi
    • Ko le lo awọn ori pupọ mọ rara
    • O nilo data ikẹkọ pupọ diẹ sii
    • A iwonba ju ni didara awoṣe
  5. Iyatọ wo ni o joko laarin boṣewa akiyesi olona-ori ati MQA?
    • Iṣọkan-Ibeere Ifojusi
    • Sisun-Window akiyesi
    • Ifarabalẹ Flash
    • Ifojusi fọnka
  6. Tani o dabaa Ifarabalẹ-Ibeere pupọ?
    • Geoffrey Hinton
    • Noam Shazeer
    • Yan LeCun
    • Andrej Karpathy
  7. In Multi-Query Attention, what remains separate for each attention head?
    • The shared key projection
    • The shared value projection
    • The query projection
    • The token sequence itself
  8. How does Grouped-Query Attention differ from standard MQA?
    • It shares query projections but gives every head separate keys and values
    • It removes attention and replaces it with a recurrent layer
    • It groups output vocabulary tokens instead of attention heads
    • It shares keys and values within several groups of query heads rather than across all heads at once