Grouped-Query Attention
Grouped-Query Attention (GQA) inzira yekumisikidza ndangariro inodiwa panguva yekugadzira zvinyorwa nekusiya misoro yemibvunzo yakati wandei igovane kiyi yakafanana nemisoro yekukosha.
Pfupiso
It makes large models much faster to serve with almost no quality loss.
Kudzika Kwakadzika
Mune yakajairwa yakawanda-yemusoro yekutarisisa layer, musoro wega wega une mibvunzo yayo, makiyi, uye kukosha. Munguva yechizvarwa, makiyi uye kukosha kune ese ekare tokens akachengetwa (iyo 'KV cache') kuti modhi irege kuvaverengera. Iine misoro mizhinji uye marefu mamiriro, iyi cache inova yakakura uye inotonga ndangariro bandwidth panguva yekufungidzira. GQA, yakaunzwa neGoogle vatsvakurudzi muna 2023, inounganidza misoro yemibvunzo uye inopa boka rega rega seti imwe chete yakagovaniswa yemakiyi nemisoro yakakosha. Kana uine misoro yemibvunzo makumi matatu nemaviri asi mapoka masere eKV chete, cache yeKV inodzikira zvakapetwa kana. Izvi zvinogara pakati pekutarisa kwakazara kwemusoro-wese (musoro wega wega wakaparadzana) uye kutarisisa kwemibvunzo yakawanda (imwe yakagovaniswa KV yemisoro yese), inotora yakawanda yekumhanya kweMQA uchichengeta mhando padyo nekunyatsotarisisa. Llama 2 70B uye akawanda akazotevera mamodheru akaitora.
Technical Insight
Hunhu hwekutarisisa hunoenderana zvakanyanya nekuve nekwakawanda kwakasiyana kwemibvunzo nzira, asi inobvumira kugovera makiyi uye kukosha. GQA inoshandisa iyi asymmetry: inochengeta misoro yemibvunzo yose asi inodzokorora musoro wega wega wakagovaniswa weKV mumibvunzo iri muboka rayo. Iko kuchengetwa kunouya pakufungidzira, uko KV cache ndiyo mutengi mukuru wekuyeuka bandwidth; Misoro mishoma yeKV inoreva dhata shoma yekuverenga pachiratidzo chakagadzirwa. Mamodheru anowanzo 'kudzidziswa' muchidimbu kuti ashandure iripo yakawanda-misoro yekutarisa kuita yeGQA imwe.
Strategic Impact
Kumhanya uye chiyero
Mutauro workflows inogona kufamba nekukurumidza pasina kupira kuenderana.
Svika uye svika
Inopamhidzira kupinda mumitauro yese nemataera ekutaurirana.
Sarudzo dzakajeka
Zvikwata zvinogona kupedza nguva yakawanda pakutonga uku otomatiki ichibata kudzokorora.
Ramangwana Rekutariswa Kwemuboka-Mubvunzo
GQA ikozvino yave yakajairika mumhando dzakavhurika-huremu nekuti inotengesa zvakachena mutengo mudiki wemhando yekuhwina kukuru. Tarisira kuti isanganise zvakanyanya nemamwe maitiro ekuita seFlashAttention, KV-cache quantization, uye zvirongwa zvitsva zvakadai se-multi-head latent attention inodzvinyirira cache kunyange mberi. Sezvo mahwindo emukati achikura, kudzora KV-cache saizi ichagara iri pakati dhizaini dambudziko, uye GQA-maitiro ekugovana musoro kucharamba kuri kiyi lever.
Real-World Implementation
Llama 2 70B uye Llama 3 vachishandisa GQA kushandira mamiriro akareba ane diki KV cache.
Kuderedza ndangariro yeGPU kuti hombe yekutaura modhi ikwane pane mashoma kana akachipa accelerators
Kumhanyisa chiratidzo-ne-chiratidzo chizvarwa mukugadzira APIs uko KV-cache bandwidth ndiyo bhodhoro.
Kugonesa hombe batch saizi yekushandira vashandisi vazhinji panguva imwe chete pasina kunetesa ndangariro
Njodzi & Guardrails
Chokwadi chehuroyi chinogona kupinda chinyararire mishumo, kuyerera kwetsigiro, kana tsvakiridzo.
Kunzwa nekukasira kunogona kugadzira mhedzisiro isingaenderane pane zvikumbiro zvakafanana.
Sensitive text data inogona kuburitswa kana zvidhiraivho zvisina kusimba.
Implementation Roadmap
Tsanangura chimiro chekubuda, toni, uye mhando zviyero usati waburitsa.
Mhinduro dzepasi neakavimbika masosi pese pazvine basa.
Chengetedza ongororo yekuongorora yemunhu kune yakakwira-stake zvinobuda.
Tevera maitiro ekutadza uye dzidzisazve kukurudzira kana mafambiro ebasa nguva nenguva.
Ramba Uchiongorora
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Grouped-Query Attention quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gaidhi rinotevera
Multi-Query Attention
Mibvunzo inowanzo bvunzwa
What is Grouped-Query Attention?
Grouped-Query Attention (GQA) inzira yekumisikidza ndangariro inodiwa panguva yekugadzira zvinyorwa nekusiya misoro yemibvunzo yakati wandei igovane kiyi yakafanana nemisoro yekukosha. Inoita mamodheru makuru kukurumidza kushandira pamwe nekusaita kurasikirwa kwemhando.
MuGrouped-Query Attention, chii chinogoverwa pakati peboka remisoro yemibvunzo?
GQA inochengeta misoro yemibvunzo yakaparadzana asi inoita kuti boka rega rega rigovane seti imwe yemakiyi nemisoro yakakosha, ichideredza cache yeKV.
GQA inotsanangurwa zvakanyanya senzvimbo yepakati pakati pezvipi zviviri zvakanyanyisa?
Multi-head inopa musoro wega wega KV; multi-query inogovera imwe KV yemisoro yose; GQA inogara pakati nemapoka mashoma eKV.
Ndeipi chikamu chekufungidzira icho GQA inonyanya kuita yakachipa?
Iko kuchengetwa kunoratidza panguva yechizvarwa, uko kuverenga iyo KV cache ndiyo yakanyanya ndangariro-bandwidth mutengo.
Kana modhi iine misoro yemibvunzo makumi matatu nemaviri uye mapoka masere eKV, cache yeKV inoderedzwa neicho chinhu?
32 query heads yakakamurwa ne8 yakagovaniswa mapoka eKV inopa zvakapetwa kuderedzwa kwakapetwa mana mumakiyi akavharirwa uye kukosha.
Nei GQA ichigona kuchengetedza huwandu hwemhando yemodhi kunyangwe kugovana misoro yeKV?
Kungwarira kunobvumira kugovera makiyi uye kukosha zviri nani pane kushivirira kurasikirwa negwara remibvunzo, saka GQA inochengeta mhando yepamusoro.