Èdè AI Itọsọna
Iṣọkan-Ibeere Ifojusi
Grouped-Query Attention (GQA) shares key/value heads across groups of query heads.
Gbọ si itọsọna yiiAI-ṣe afihan · ~1 min
Lori iwe yi1 iseju kika
Akopọ
The original GQA paper studies this as a tradeoff between multi-head and multi-query attention for decoder inference.
Jin Dive
The GQA paper proposes grouping query heads so multiple query heads share key/value heads. MHA has one key/value head per query head; MQA shares a single key/value head across all queries; GQA lies between them. The paper evaluates GQA on language modeling and machine translation tasks and reports quality closer to MHA with inference speed closer to MQA in its experiments. It also reports uptraining existing MHA models using about 5% of original pretraining compute. These claims are limited to the paper’s models, tasks and setup.
Real-World imuse
Comparing MHA, GQA and MQA head sharing.
Estimating KV-cache size from the number of query heads and key/value groups.
Reading the paper’s reported quality/speed tradeoff on evaluated language-model tasks.
Reviewing the paper’s uptraining recipe for converting an MHA checkpoint to GQA.
Ojo iwaju ti Ifarabalẹ-Ibeere Ẹgbẹ
Benchmark head-sharing choices on the target model and hardware. The original paper’s quality/speed results do not establish universal serving speedups or negligible quality loss across all systems.
Fifi o sinu iwa
Kini idi ti o ṣe pataki
- Iyara ati iwọn. Ṣiṣan iṣẹ ede le gbe ni iyara laisi irubọ aitasera.
- Wiwọle ati arọwọto. O faagun iraye si kọja awọn ede ati awọn aza ibaraẹnisọrọ.
- Awọn ipinnu diẹ sii. Awọn ẹgbẹ le lo akoko diẹ sii lori idajọ lakoko ti adaṣe n kapa atunwi.
Ṣọra fun
- Awọn otitọ ti a sọ di mimọ le tẹ awọn ijabọ sii ni idakẹjẹ, awọn ṣiṣan atilẹyin, tabi awọn abajade iwadii.
- Ifamọ kiakia le ṣẹda awọn abajade aisedede kọja awọn ibeere ti o jọra.
- Awọn data ọrọ ifarabalẹ le farahan ti awọn idari wiwọle ko lagbara.
Next awọn igbesẹ
- Ṣetumo ọna kika iṣẹjade, ohun orin, ati awọn iṣedede didara ṣaaju ṣiṣejade.
- Awọn idahun ilẹ pẹlu awọn orisun ti o gbẹkẹle nigbakugba ti deede ba ṣe pataki.
- Jeki aaye ayẹwo atunyẹwo eniyan fun awọn abajade ti o ga julọ.
- Tọpinpin awọn ilana ikuna ati tunṣe awọn itọsi tabi ṣiṣan iṣẹ nigbagbogbo.
Tesiwaju Ṣiṣawari
Ìwé ìròyìn ọfẹ
Gba ìkànìyàn AI ojoojúmọ́
Ìtàn AI mẹta tó jẹ́rìí ní gbogbo owurọ́ ọjọ́ iṣẹ́, kọ́wé ní Gẹ̀ẹ́sì tó rọrùn. Ọfẹ́ fún gbogbo àkókò, kò sí ìpolówó.
Ìmeèlì kan ní gbogbo ọjọ́ iṣẹ́. Yọkúrò ní ìkànìyàn ní ìkànìyàn kan. A kò ta tàbí pín àdírẹ́sì rẹ.
Ṣe idanwo fun ara rẹ
Take the Grouped-Query Attention quiz
Ìdáhùn lẹ́sẹ̀kẹsẹ̀ lórí gbogbo ìdáhùn, àti iwe-ẹ̀rí tó lè pin pẹ̀lú ID tó lè jẹ́rìí lẹ́yìn tí o bá kọ́ ẹ̀kọ́.
Ṣe àtìlẹ́yìn ẹ̀kọ́ AI ọfẹ́. AI Understanding jẹ́ àjọ àìní èrè 501(c)(3) — kò sí ìpolówó, kò sí àkúnya owó, ní gbogbo àkókò. Ṣe ẹbun kan
Awọn ibeere ti a beere nigbagbogbo
Kini Ifarabalẹ-Ibeere Ẹgbẹ?
Grouped-Query Attention (GQA) shares key/value heads across groups of query heads. The original GQA paper studies this as a tradeoff between multi-head and multi-query attention for decoder inference.
Ninu Ifarabalẹ-Ibeere ti Akojọpọ, kini a pin laarin ẹgbẹ kan ti awọn olori ibeere?
GQA tọju awọn ori ibeere lọtọ ṣugbọn jẹ ki ẹgbẹ kọọkan wọn pin ipin kan ti bọtini ati awọn ori iye, idinku kaṣe KV.
GQA ti wa ni ti o dara ju apejuwe bi a aarin laarin eyi ti meji extremes?
Olona-ori yoo fun gbogbo ori KV tirẹ; olona-ibeere pin ọkan KV fun gbogbo awọn olori; GQA joko laarin pẹlu awọn ẹgbẹ KV diẹ.
Kini apakan ti itọkasi GQA ni akọkọ jẹ din owo?
Awọn ifowopamọ ṣe afihan ni akoko iran, nibiti kika kaṣe KV jẹ idiyele iranti-bandwidth ti o ga julọ.
Ti awoṣe ba ni awọn ori ibeere 32 ati awọn ẹgbẹ KV 8, kaṣe KV dinku nipasẹ aijọju kini ifosiwewe?
Awọn ori ibeere 32 ti o pin nipasẹ awọn ẹgbẹ KV 8 ti o pin yoo funni ni aijọju idinku mẹrin ni awọn bọtini ipamọ ati awọn iye.
What quality/speed tradeoff did the GQA paper report in its experiments?
The GQA paper reports quality closer to multi-head attention (MHA) with inference speed closer to multi-query attention (MQA) in its evaluated tasks. This is an empirical result, not a universal guarantee.
Tesiwaju kikọ
Jẹmọ awọn itọsọna
Awọn itọsọna diẹ sii ti a yan fun koko yii