Làkk AI GUIDE
Laajte buñ boole
Grouped-Query Attention (GQA) shares key/value heads across groups of query heads.
Déglul tegtal biiIA-nettali · ~1 min
Ci xët wii1 simili jàng
Résumé
The original GQA paper studies this as a tradeoff between multi-head and multi-query attention for decoder inference.
Plongeur bu xóot
The GQA paper proposes grouping query heads so multiple query heads share key/value heads. MHA has one key/value head per query head; MQA shares a single key/value head across all queries; GQA lies between them. The paper evaluates GQA on language modeling and machine translation tasks and reports quality closer to MHA with inference speed closer to MQA in its experiments. It also reports uptraining existing MHA models using about 5% of original pretraining compute. These claims are limited to the paper’s models, tasks and setup.
Doxal ci àdduna dëgg
Comparing MHA, GQA and MQA head sharing.
Estimating KV-cache size from the number of query heads and key/value groups.
Reading the paper’s reported quality/speed tradeoff on evaluated language-model tasks.
Reviewing the paper’s uptraining recipe for converting an MHA checkpoint to GQA.
Ëlëgu bàyyi xel ci laaj yuñ boole
Benchmark head-sharing choices on the target model and hardware. The original paper’s quality/speed results do not establish universal serving speedups or negligible quality loss across all systems.
Teg ko ci jëf
Lu tax mu am solo
- Gaawaay ak yaatuwaay. Liggéeyukaay yi ci làkk yi mën nañu gëna gaaw te duñu yàq deggoo gi.
- Dugg ak yegg. Dafay yaatal jëfandikoo gi ci làkk yi ak ci anam yi ñuy jokkoo.
- dogal yu gëna leer. Ekip yi mën nañu gëna yàgg ci àtte ci jamono ji otomatisation di liggéey ci baamtu.
Moytul
- Lépp lu jaarul yoon mën na dugg ci rapoor yi, jàppale ci liggéey bi, wala ci njariñu gëstu bi.
- Sensibilite bu gaaw mën na jur njariñ yu wuute ci laajte yu noonu mel.
- Done yu am solo mën nañu feeñ sudee seytu jëfandikoo gi néew doole.
Jéego yi ci topp
- Mandargal formaa génne gi, melokaan bi, ak standard kalite yi laata ngay dugal ko.
- Tontu yu am solo ak balluwaay yu wóor saa yu dëggu bi di am solo.
- Fexeel am barabu xool nit ñi ngir am njariñ yu am solo.
- Toppal anami gacce yi ak di faral di tàggataat ay laaj wala def-liggéey.
Weyal di banneexu
Bulletin bu amul fayda
Jàngal xibaar AI bu bes
Ñatti jaar-jaari IA yuñ firndeel bis bu nekk ci ayu-bis bi, ñu bind leen ci Àngle bu leer. Gratis ba fàww, amul yëgle.
Benn email bis bu nekk ci ayu-bis bi. Benn klik nga dindi abonemaa bi. Dunu musa jaay wala séddoo sa adres.
Nattal sa bopp
Take the Grouped-Query Attention quiz
Feedback ci saasi ci bépp tontu, ak sertifikaa buñ mëna séddoo ak ID buñ mëna saytu soo jàllee ci njàng mi.
Jàppale njàngum IA te doo fay. AI Understanding mbootaay la buy def te Yàlla tax — amul yëgle, amul paywall, musul am. Defal ab sarax
Laaj yi ñuy faral di laaj
Luy Fexe Laaj buñ boole?
Grouped-Query Attention (GQA) shares key/value heads across groups of query heads. The original GQA paper studies this as a tradeoff between multi-head and multi-query attention for decoder inference.
Ci Grouped-Query Attention, lan lañu bokk ci mbooloo mu boppu laaj yi?
GQA dafay tàqale boppu laaj yi waaye dafay may kuréel gu nekk ci ñoom ñu bokk benn boppu caabi ak valeur, di wàññi cache KV.
GQA mooy gëna mëna wax ni diggu suuf la ci diggante ban ñaari extrême?
Bopp bu bari dafay jox bopp bu nekk KV boppam; multi-query bokk benn KV ngir bopp yépp; GQA mingi toog ci digg bi ak ay kuréel KV yu néew.
Ban wàll ci inference la GQA gëna xéewale?
Sakkanal bi dafay wane ci jamonoy defar, fu jàng cache KV mooy njëgu bandwidth memory bi gëna am solo.
Sudee benn model amna 32 boppu laaj ak 8 grupu KV, cache KV bi dafay wàññeeku ci lu tollu ci ban facteur?
32 boppu laajte xaaj 8 grupu KV yuñ bokk dafay joxe lu tollu ci ñeenti yoon wàññeeku ci caabi ak valeur yiñ denc.
What quality/speed tradeoff did the GQA paper report in its experiments?
The GQA paper reports quality closer to multi-head attention (MHA) with inference speed closer to multi-query attention (MQA) in its evaluated tasks. This is an empirical result, not a universal guarantee.
Weyal di jàng
Gid yu jëm ci loolu
Tann nañu yeneen njiit ngir topic bii