InoteveraGaidhi rinotevera
Multi-Query Attention
Mutauro AI
Mutauro AI GUIDE
Grouped-Query Attention
Grouped-Query Attention (GQA) shares key/value heads across groups of query heads.
Teerera kugwaro iriAI-yakarondedzerwa · ~1 min
Pa peji ino1 min verenga
Pfupiso
The original GQA paper studies this as a tradeoff between multi-head and multi-query attention for decoder inference.
Kudzika Kwakadzika
The GQA paper proposes grouping query heads so multiple query heads share key/value heads. MHA has one key/value head per query head; MQA shares a single key/value head across all queries; GQA lies between them. The paper evaluates GQA on language modeling and machine translation tasks and reports quality closer to MHA with inference speed closer to MQA in its experiments. It also reports uptraining existing MHA models using about 5% of original pretraining compute. These claims are limited to the paper’s models, tasks and setup.
Kuitwa muNyika yeChokwadi
Comparing MHA, GQA and MQA head sharing.
Estimating KV-cache size from the number of query heads and key/value groups.
Reading the paper’s reported quality/speed tradeoff on evaluated language-model tasks.
Reviewing the paper’s uptraining recipe for converting an MHA checkpoint to GQA.
Ramangwana Rekutariswa Kwemuboka-Mubvunzo
Benchmark head-sharing choices on the target model and hardware. The original paper’s quality/speed results do not establish universal serving speedups or negligible quality loss across all systems.
Kuzvishandisa
Nei zvichikosha
- Kumhanya uye chiyero. Mutauro workflows inogona kufamba nekukurumidza pasina kupira kuenderana.
- Svika uye svika. Inopamhidzira kupinda mumitauro yese nemataera ekutaurirana.
- Sarudzo dzakajeka. Zvikwata zvinogona kupedza nguva yakawanda pakutonga uku otomatiki ichibata kudzokorora.
Chenjera
- Chokwadi chehuroyi chinogona kupinda chinyararire mishumo, kuyerera kwetsigiro, kana tsvakiridzo.
- Kunzwa nekukasira kunogona kugadzira mhedzisiro isingaenderane pane zvikumbiro zvakafanana.
- Sensitive text data inogona kuburitswa kana zvidhiraivho zvisina kusimba.
Matanho anotevera
- Tsanangura chimiro chekubuda, toni, uye mhando zviyero usati waburitsa.
- Mhinduro dzepasi neakavimbika masosi pese pazvine basa.
- Chengetedza ongororo yekuongorora yemunhu kune yakakwira-stake zvinobuda.
- Tevera maitiro ekutadza uye dzidzisazve kukurudzira kana mafambiro ebasa nguva nenguva.
Ramba Uchiongorora
Tsamba yemahara
Tora iyo yemazuva ese AI pfupiso
Nhatu dzakasimbiswa dzeAI nyaya mangwanani ega ega pakati pevhiki, dzakanyorwa neChirungu chakajeka. Yemahara zvachose, hapana ads.
Imwe email zuva rega rega revhiki. Bvisa kunyoresa kamwe chete. Hatimbotengese kana kugovera kero yako.
Zviedze iwe pachako
Take the Grouped-Query Attention quiz
Mhinduro yepakarepo pamhinduro yega yega, uye chitupa chinogovaniswa chine ID inobatika kana wapedza kosi.
Tsigira dzidzo yemahara yeAI. AI Understanding ndeye 501(c)(3) isingabatsiri - hapana ads, hapana paywall, zvachose. Ita chipo
Mibvunzo inowanzo bvunzwa
What is Grouped-Query Attention?
Grouped-Query Attention (GQA) shares key/value heads across groups of query heads. The original GQA paper studies this as a tradeoff between multi-head and multi-query attention for decoder inference.
MuGrouped-Query Attention, chii chinogoverwa pakati peboka remisoro yemibvunzo?
GQA inochengeta misoro yemibvunzo yakaparadzana asi inoita kuti boka rega rega rigovane seti imwe yemakiyi nemisoro yakakosha, ichideredza cache yeKV.
GQA inotsanangurwa zvakanyanya senzvimbo yepakati pakati pezvipi zviviri zvakanyanyisa?
Multi-head inopa musoro wega wega KV; multi-query inogovera imwe KV yemisoro yose; GQA inogara pakati nemapoka mashoma eKV.
Ndeipi chikamu chekufungidzira icho GQA inonyanya kuita yakachipa?
Iko kuchengetwa kunoratidza panguva yechizvarwa, uko kuverenga iyo KV cache ndiyo yakanyanya ndangariro-bandwidth mutengo.
Kana modhi iine misoro yemibvunzo makumi matatu nemaviri uye mapoka masere eKV, cache yeKV inoderedzwa neicho chinhu?
32 query heads yakakamurwa ne8 yakagovaniswa mapoka eKV inopa zvakapetwa kuderedzwa kwakapetwa mana mumakiyi akavharirwa uye kukosha.
What quality/speed tradeoff did the GQA paper report in its experiments?
The GQA paper reports quality closer to multi-head attention (MHA) with inference speed closer to multi-query attention (MQA) in its evaluated tasks. This is an empirical result, not a universal guarantee.
Ramba uchidzidza
Mazano anoenderana
Mamwe madhairekitori akasarudzirwa nyaya iyi