Mutauro AI GUIDE

Multi-Query Attention

Multi-Query Attention (MQA) inzira inochengetedza ndangariro pakutarisisa kwetransformer iyo inogovera seti imwe yemakiyi uye kukosha pamisoro yese yekutarisisa.

Teerera kugwaro iriAI-yakarondedzerwa · ~2 min

Tanga kuverengaTora mubvunzo
Pa peji ino2 min verenga
  1. Pfupiso
  2. Kudzika Kwakadzika
  3. Kuitwa muNyika yeChokwadi
  4. Ramangwana reMulti-Query Attention
  5. Kuzvishandisa
  6. Ramba Uchiongorora
  7. Mibvunzo inowanzo bvunzwa

Pfupiso

Iyo inomhanyisa zvinoshamisa kugadzira zvinyorwa nekudzikamisa ndangariro iyo modhi inofanirwa kutenderera ichitenderedza.

Kudzika Kwakadzika

Yakajairwa yakawanda-yemusoro kutarisa inopa musoro wega wega mubvunzo, kiyi, uye kukosha kwekufungidzira. Munguva yechizvarwa, makiyi uye kukosha kwezvese tokens zvakapfuura zvinofanirwa kuvharirwa uye kurodha padanho rega rega - iyi KV cache inova iyo huru yebhodhoro, sezvo kuiverenga kubva mundangariro kunononoka kupfuura iyo masvomhu pachayo. Multi-Query Attention, yakakurudzirwa naNoam Shazeer muna 2019, inochengeta fungidziro yemubvunzo wakasiyana pamusoro asi inopunzika makiyi uye kukosha kune imwechete yakagovaniswa musoro. Izvi zvinoderedza cache yeKV nechinhu chakaenzana nenhamba yemisoro, dzimwe nguva 8x kusvika 64x diki. Mhedzisiro yacho inokurumidza kukurumidza autoregressive decoding uye yakareruka ndangariro tsoka, ine chete ine mwero mhando dip. Nzvimbo yepakati, Grouped-Query Attention, inoenzanisa kutengeserana.

Kuitwa muNyika yeChokwadi

Kumhanyisa chizvarwa chechiratidzo-ne-chiratidzo muvabatsiri vekutaura uko KV cache, kwete mbishi komputa, inodzikamisa kubuda.

Google's PaLM, iyo yakashandisa Multi-Query Attention kuti igone kugonesa fungidziro huru.

Kushandira vashandisi vazhinji panguva imwe chete pane imwe GPU nekudzikisa iyo-yega-chikumbiro KV cache memory.

Grouped-Query Attention muLlama 2 70B uye Llama 3, dzinza rakananga rinoenzanisa kumhanya kweMQA nekunyatsotarisisa mhando.

Ramangwana reMulti-Query Attention

MQA yakasimbisa kuti iwe unogona kuchekerera makiyi akawandisa / kukosha misoro nekukuvadza kushoma, uye nzwisiso iko zvino inoumba dzinenge dzese dzekukurumidza-inference LLM. Munda wakanyanya kuungana paGrouped-Query Attention (GQA), inoshandiswa muLlama 2/3 nevamwe vazhinji, iyo inoshandisa mashoma emapoka eKV pane imwe kudzoreredza mhando uchichengeta yakawanda yekumhanyisa. Basa remangwana rinosanganisa aya mazano neKV-cache compression, quantization, uye yakawanda-yanonoka kutarisisa kusundira kureba mamiriro uye zvakachipa kushumira.

Kuzvishandisa

Nei zvichikosha

  • Kumhanya uye chiyero. Mutauro workflows inogona kufamba nekukurumidza pasina kupira kuenderana.
  • Svika uye svika. Inopamhidzira kupinda mumitauro yese nemataera ekutaurirana.
  • Sarudzo dzakajeka. Zvikwata zvinogona kupedza nguva yakawanda pakutonga uku otomatiki ichibata kudzokorora.

Chenjera

  • Chokwadi chehuroyi chinogona kupinda chinyararire mishumo, kuyerera kwetsigiro, kana tsvakiridzo.
  • Kunzwa nekukasira kunogona kugadzira mhedzisiro isingaenderane pane zvikumbiro zvakafanana.
  • Sensitive text data inogona kuburitswa kana zvidhiraivho zvisina kusimba.

Matanho anotevera

  1. Tsanangura chimiro chekubuda, toni, uye mhando zviyero usati waburitsa.
  2. Mhinduro dzepasi neakavimbika masosi pese pazvine basa.
  3. Chengetedza ongororo yekuongorora yemunhu kune yakakwira-stake zvinobuda.
  4. Tevera maitiro ekutadza uye dzidzisazve kukurudzira kana mafambiro ebasa nguva nenguva.

Ramba Uchiongorora

ChatGPT neLLMs

Ona kuti mamodheru emitauro yemazuva ano anoburitsa uye kufunga sei.Verenga Guide

Nheyo dzeNLP

Dzidza zvakakosha zvekugadzirisa mutauro kuseri kwezvishandiso izvi.Verenga Guide

Tsamba yemahara

Tora iyo yemazuva ese AI pfupiso

Nhatu dzakasimbiswa dzeAI nyaya mangwanani ega ega pakati pevhiki, dzakanyorwa neChirungu chakajeka. Yemahara zvachose, hapana ads.

Imwe email zuva rega rega revhiki. Bvisa kunyoresa kamwe chete. Hatimbotengese kana kugovera kero yako.

Zviedze iwe pachako

Tora iyo Multi-Query Attention quiz

Mhinduro yepakarepo pamhinduro yega yega, uye chitupa chinogovaniswa chine ID inobatika kana wapedza kosi.

Tanga mibvunzo

Tsigira dzidzo yemahara yeAI. AI Understanding ndeye 501(c)(3) isingabatsiri - hapana ads, hapana paywall, zvachose. Ita chipo

Mibvunzo inowanzo bvunzwa

Chii chinonzi Multi-Query Attention?

Multi-Query Attention (MQA) inzira inochengetedza ndangariro pakutarisisa kwetransformer iyo inogovera seti imwe yemakiyi uye kukosha pamisoro yese yekutarisisa. Iyo inomhanyisa zvinoshamisa kugadzira zvinyorwa nekudzikamisa ndangariro iyo modhi inofanirwa kutenderera ichitenderedza.

Chii chinogovera Multi-Query Attention pamisoro yese yekutarisisa?

MQA inochengeta yakaparadzana mibvunzo pamusoro wega asi inogovanisa kiyi yekufungidzira uye imwe kukosha kwekufungidzira pamisoro yese.

Ndeipi yekutanga bhodhoro iyo MQA inogadzirisa panguva yekugadzira autoregressive?

Kudzoreredza iyo yakakura KV cache nhanho imwe neimwe ndeye bandwidth-yakasungwa; MQA inodzikisa cache iyoyo kuti ikurumidze kudhikodha.

Nekuda kwechinhu chipi chinoita kuti MQA idzore cache yeKV?

Sezvo makiyi nemakoshero achidonha kubva kuH misoro kuenda kune imwe, cache inodzikira neinenge kuverenga musoro.

Ndeipi kutengeserana kukuru kwekushandisa MQA?

Kugovera makiyi uye kukosha kunoderedza kumiririra simba zvishoma, zvichikonzera kuderera kwemhando mukutsinhana kwekuwana kukuru kwekumhanya.

Ndeupi musiyano unogara pakati peyakajairwa-yepamusoro-soro kutarisa uye MQA?

Grouped-Query Attention (GQA) inoshandisa akati wandei maKV mapoka pachinzvimbo cheimwe, kuenzanisa mhando nekumhanya.

Ramba uchidzidza

Mazano anoenderana

Mamwe madhairekitori akasarudzirwa nyaya iyi

Wana chitupa cheAI chemaharaKosi pfupi yemahara nebvunzo. Hapana mutengo, hapana kadhi. Inoda kusvika maawa 2.5.Tanga mahara →