Multi-Query Attention
Multi-Query Attention (MQA) inzira inochengetedza ndangariro pakutarisisa kwetransformer iyo inogovera seti imwe yemakiyi uye kukosha pamisoro yese yekutarisisa.
Pfupiso
It dramatically speeds up text generation by shrinking the memory the model must shuffle around.
Kudzika Kwakadzika
Yakajairwa yakawanda-yemusoro kutarisa inopa musoro wega wega mubvunzo, kiyi, uye kukosha kwekufungidzira. Munguva yechizvarwa, makiyi uye kukosha kwezvese tokens zvakapfuura zvinofanirwa kuvharirwa uye kurodha padanho rega rega - iyi KV cache inova iyo huru yebhodhoro, sezvo kuiverenga kubva mundangariro kunononoka kupfuura iyo masvomhu pachayo. Multi-Query Attention, yakakurudzirwa naNoam Shazeer muna 2019, inochengeta fungidziro yemubvunzo wakasiyana pamusoro asi inopunzika makiyi uye kukosha kune imwechete yakagovaniswa musoro. Izvi zvinoderedza cache yeKV nechinhu chakaenzana nenhamba yemisoro, dzimwe nguva 8x kusvika 64x diki. Mhedzisiro yacho inokurumidza kukurumidza autoregressive decoding uye yakareruka ndangariro tsoka, ine chete ine mwero mhando dip. Nzvimbo yepakati, Grouped-Query Attention, inoenzanisa kutengeserana.
Technical Insight
MuMQA, uremu hwemubvunzo huchiri kuburitsa H akapatsanurwa query vectors, asi imwechete kiyi fungidziro uye imwechete kukosha fungidziro inogovaniswa pamisoro yese. Musoro wega wega unoverengera kutarisa uchishandisa muvhunzo wawo uchipesana nemakiyi nemakoshero akafanana. Nekuti iyo cached K neV tensor haichayereki nenhamba yemisoro, ndangariro bandwidth panguva yedecoding inodonha zvakanyanya - uye bandwidth, kwete compute, ndiyo iyo magedhi echizvarwa kumhanya pane ano accelerators.
Strategic Impact
Kumhanya uye chiyero
Mutauro workflows inogona kufamba nekukurumidza pasina kupira kuenderana.
Svika uye svika
Inopamhidzira kupinda mumitauro yese nemataera ekutaurirana.
Sarudzo dzakajeka
Zvikwata zvinogona kupedza nguva yakawanda pakutonga uku otomatiki ichibata kudzokorora.
Ramangwana reMulti-Query Attention
MQA yakasimbisa kuti iwe unogona kuchekerera makiyi akawandisa / kukosha misoro nekukuvadza kushoma, uye nzwisiso iko zvino inoumba dzinenge dzese dzekukurumidza-inference LLM. Munda wakanyanya kuungana paGrouped-Query Attention (GQA), inoshandiswa muLlama 2/3 nevamwe vazhinji, iyo inoshandisa mashoma emapoka eKV pane imwe kudzoreredza mhando uchichengeta yakawanda yekumhanyisa. Basa remangwana rinosanganisa aya mazano neKV-cache compression, quantization, uye yakawanda-yanonoka kutarisisa kusundira kureba mamiriro uye zvakachipa kushumira.
Real-World Implementation
Kumhanyisa chizvarwa chechiratidzo-ne-chiratidzo muvabatsiri vekutaura uko KV cache, kwete mbishi komputa, inodzikamisa kubuda.
Google's PaLM, iyo yakashandisa Multi-Query Attention kuti igone kugonesa fungidziro huru.
Kushandira vashandisi vazhinji panguva imwe chete pane imwe GPU nekudzikisa iyo-yega-chikumbiro KV cache memory.
Grouped-Query Attention muLlama 2 70B uye Llama 3, dzinza rakananga rinoenzanisa kumhanya kweMQA nekunyatsotarisisa mhando.
Njodzi & Guardrails
Chokwadi chehuroyi chinogona kupinda chinyararire mishumo, kuyerera kwetsigiro, kana tsvakiridzo.
Kunzwa nekukasira kunogona kugadzira mhedzisiro isingaenderane pane zvikumbiro zvakafanana.
Sensitive text data inogona kuburitswa kana zvidhiraivho zvisina kusimba.
Implementation Roadmap
Tsanangura chimiro chekubuda, toni, uye mhando zviyero usati waburitsa.
Mhinduro dzepasi neakavimbika masosi pese pazvine basa.
Chengetedza ongororo yekuongorora yemunhu kune yakakwira-stake zvinobuda.
Tevera maitiro ekutadza uye dzidzisazve kukurudzira kana mafambiro ebasa nguva nenguva.
Ramba Uchiongorora
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Multi-Query Attention quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gaidhi rinotevera
Grouped-Query Attention
Mibvunzo inowanzo bvunzwa
What is Multi-Query Attention?
Multi-Query Attention (MQA) inzira inochengetedza ndangariro pakutarisisa kwetransformer iyo inogovera seti imwe yemakiyi uye kukosha pamisoro yese yekutarisisa. Iyo inomhanyisa zvinoshamisa kugadzira zvinyorwa nekudzikamisa ndangariro iyo modhi inofanirwa kutenderera ichitenderedza.
Chii chinogovera Multi-Query Attention pamisoro yese yekutarisisa?
MQA inochengeta yakaparadzana mibvunzo pamusoro wega asi inogovanisa kiyi yekufungidzira uye imwe kukosha kwekufungidzira pamisoro yese.
Ndeipi yekutanga bhodhoro iyo MQA inogadzirisa panguva yekugadzira autoregressive?
Kudzoreredza iyo yakakura KV cache nhanho imwe neimwe ndeye bandwidth-yakasungwa; MQA inodzikisa cache iyoyo kuti ikurumidze kudhikodha.
Nekuda kwechinhu chipi chinoita kuti MQA idzore cache yeKV?
Sezvo makiyi nemakoshero achidonha kubva kuH misoro kuenda kune imwe, cache inodzikira neinenge kuverenga musoro.
Ndeipi kutengeserana kukuru kwekushandisa MQA?
Kugovera makiyi uye kukosha kunoderedza kumiririra simba zvishoma, zvichikonzera kuderera kwemhando mukutsinhana kwekuwana kukuru kwekumhanya.
Ndeupi musiyano unogara pakati peyakajairwa-yepamusoro-soro kutarisa uye MQA?
Grouped-Query Attention (GQA) inoshandisa akati wandei maKV mapoka pachinzvimbo cheimwe, kuenzanisa mhando nekumhanya.