Luqadda AI HAGAHA
Feejignaan-kooxeed
Grouped-Query Attention (GQA) shares key/value heads across groups of query heads.
Dhageyso hagahanAI-waa laga sheekeeyay · ~1 min
Boggaan1 daqiiqo akhri
Dulmar
The original GQA paper studies this as a tradeoff between multi-head and multi-query attention for decoder inference.
quusid qoto dheer
The GQA paper proposes grouping query heads so multiple query heads share key/value heads. MHA has one key/value head per query head; MQA shares a single key/value head across all queries; GQA lies between them. The paper evaluates GQA on language modeling and machine translation tasks and reports quality closer to MHA with inference speed closer to MQA in its experiments. It also reports uptraining existing MHA models using about 5% of original pretraining compute. These claims are limited to the paper’s models, tasks and setup.
Dhaqangelinta Adduunka-dhabta ah
Comparing MHA, GQA and MQA head sharing.
Estimating KV-cache size from the number of query heads and key/value groups.
Reading the paper’s reported quality/speed tradeoff on evaluated language-model tasks.
Reviewing the paper’s uptraining recipe for converting an MHA checkpoint to GQA.
Mustaqbalka Feejignaanta Su'aalaha kooxaysan
Benchmark head-sharing choices on the target model and hardware. The original paper’s quality/speed results do not establish universal serving speedups or negligible quality loss across all systems.
Ku dhaqangelinta
Maxay muhiim u tahay
- Xawaaraha iyo miisaanka. Socodka shaqada luqaddu si dhakhso leh ayay u socon kartaa iyada oo aan la hurayn joogteynta.
- Helitaanka iyo gaarsiinta. Waxay balaadhisaa gelitaanka luqadaha iyo qaababka isgaarsiinta.
- Go'aamo cad. Kooxuhu waxay waqti badan ku qaadan karaan xukunka halka otomaatiggu uu qabanayo ku celcelinta.
Iska ilaali
- Xaqiiqooyinka dhalanteed waxay si deggan u geli karaan warbixinnada, taageerada socodka, ama natiijooyinka cilmi-baarista.
- Dareenka degdega ahi wuxuu abuuri karaa natiijooyin aan iswaafaqayn codsiyada la midka ah.
- Xogta qoraalka xasaasiga ah ayaa laga yaabaa in la kashifo haddii kontaroolada gelitaanka ay daciif yihiin.
Tallaabooyinka xiga
- Qeex qaabka wax soo saarka, codka, iyo heerarka tayada ka hor inta aan la baahin.
- Jawaabaha salka ku haya ilo lagu kalsoon yahay mar kasta oo saxnidu ay muhiim tahay.
- Hayso isbaarada dib u eegista bini aadamka ee wax soo saarka sare.
- Lasoco qaababka guuldarada oo dib u leyli dardargelinta ama socodka shaqada si joogto ah.
Sii wad Sahaminta
Warqad bilaash ah
Hel warbixinta maalinlaha ah ee AI
Saddex sheeko oo la xaqiijiyay oo AI ah maalin kasta oo shaqo ah, oo lagu qoray Ingiriis fudud. Bilaash had iyo jeer, aan xayaysiis lahayn.
Hal iimayl maalin kasta oo shaqo ah. Ka bax hal gujin. Marna ma iibinno ama ma wadaagno ciwaankaaga.
Is tijaabi
Take the Grouped-Query Attention quiz
Fikrad degdeg ah oo ku saabsan jawaab kasta, iyo shahaado la wadaajin karo oo leh aqoonsi la xaqiijin karo markaad ka gudubto koorso.
Taageer waxbarashada AI bilaashka ah. AI Understanding waa 501(c)(3) aan faa'iido doon ahayn — xayaysiis la'aan, albaab lacag ah ma jiro, had iyo jeer. Samee deeq
Su'aalaha soo noqnoqda
What is Grouped-Query Attention?
Grouped-Query Attention (GQA) shares key/value heads across groups of query heads. The original GQA paper studies this as a tradeoff between multi-head and multi-query attention for decoder inference.
Feejignaan-kooxeed-Su'aal, maxaa la wadaagaa kooxda madaxda weydiinta?
GQA waxay haysaa madax su'aalo gaar ah laakiin waxay u ogolaataa koox kasta oo iyaga ka mid ah inay wadaagaan hal xidhmo oo furayaal ah iyo madax qiimayaal ah, iyagoo yaraynaya kaydka KV.
GQA waxaa sida ugu wanaagsan loogu tilmaamay meel dhexe oo u dhaxaysa labada daraf?
Madax-badan ayaa madax walba siinaya KV u gaar ah; su'aalo badan ayaa wadaagaya hal KV dhammaan madaxda; GQA waxay dhex fadhidaa dhawr kooxood oo KV ah.
Waa maxay qayb ka mid ah fikradda GQA ugu horrayn ka jaban tahay?
Kaydinta waxay soo baxdaa wakhtiga jiilka, halkaasoo akhrinta khasnadda KV ay tahay qiimaha ugu badan ee xusuusta-bandwidth-ka.
Haddii moodal leeyahay 32 madax waydiimo iyo 8 kooxood oo KV ah, khasnadda KV waa la dhimay qiyaastee?
32 madax su'aalood oo ay u qaybiyeen 8 kooxood oo KV ah oo la wadaago ayaa ku dhawaad afar laab hoos u dhac ku keenaya furayaasha iyo qiyamka la kaydiyay.
What quality/speed tradeoff did the GQA paper report in its experiments?
The GQA paper reports quality closer to multi-head attention (MHA) with inference speed closer to multi-query attention (MQA) in its evaluated tasks. This is an empirical result, not a universal guarantee.
Sii wad waxbarashada
Tilmaamaha la xidhiidha
Tilmaamayaal badan ayaa loo doortay mawduucan