Nlebanya Ajụjụ agbakọtara
Nleba anya ajụjụ ọnụ (GQA) bụ ụzọ isi belata ebe nchekwa achọrọ n'oge ọgbọ ederede site n'ikwe ka ọtụtụ ndị isi ajụjụ kesaa otu igodo na isi uru.
Nchịkọta
It makes large models much faster to serve with almost no quality loss.
Ime miri emi
N'ọkwa ọkọlọtọ nlebara anya ọtụtụ isi, isi ọ bụla nwere ajụjụ, igodo, na ụkpụrụ nke ya. N'oge ọgbọ, igodo na ụkpụrụ maka akara ngosi niile gara aga na-echekwa (' cache 'KV') ka ihe nlereanya ahụ ghara ịgbakọ ha. Na ọtụtụ isi na ogologo okirikiri, cache a na-aghọ nnukwu ma na-achịkwa bandwit ebe nchekwa n'oge ntinye. GQA, nke ndị nyocha Google webatara na 2023, na-achịkọta ndị isi ajụjụ wee nye otu ọ bụla otu isi igodo na isi uru. Ọ bụrụ na ị nwere isi ajụjụ 32 mana naanị otu KV 8, cache KV na-agbada ihe ruru okpukpu anọ. Nke a na-anọdụ n'etiti nlebara anya multi-isi zuru oke (isi ọ bụla dị iche iche) na nlebara anya ọtụtụ ajụjụ (otu kesara KV maka isi niile), na-ewepụta ọtụtụ ọsọ nke MQA ka ị na-edobe àgwà nso na nlebara anya zuru oke. Llama 2 70B na ọtụtụ ụdị emechara nakweere ya.
Nghọta nka nka
Ogo nlebara anya na-adabere kpamkpam n'inwe ọtụtụ ntụzịaka ajụjụ dị iche, mana ọ na-anabata ikesa igodo na ụkpụrụ. GQA na-erigbu asymmetry a: ọ na-edobe isi ajụjụ niile mana ọ na-emegharị isi KV nke ọ bụla n'ime ajụjụ ndị dị na otu ya. Ihe nchekwa ahụ na-abịa na ntinye, ebe nchekwa KV bụ onye na-azụ ahịa bandwidth ebe nchekwa; isi KV ole na ole pụtara obere data ịgụta kwa akara ngosi emepụtara. A na-azụkarị ụdịdị a nwa oge iji gbanwee ebe nlele ọtụtụ isi dị ugbu a ka ọ bụrụ nke GQA.
Mmetụta atụmatụ
Ọsọ na ọnụ ọgụgụ
Usoro ọrụ asụsụ nwere ike ịga ngwa ngwa n'achụghị nkwụsi ike.
Nweta na iru
Ọ na-agbasawanye ohere n'ofe asụsụ na ụdị nzikọrịta ozi.
Mkpebi doro anya
Ndị otu nwere ike itinyekwu oge na ikpe ebe akpaaka na-ejikwa nkwughachi.
Ọdịnihu nke nlebara anya ajụjụ ọnụ agbakọtara
GQA bụ ọkọlọtọ ọkọlọtọ ugbu a na ụdị dị arọ mepere emepe n'ihi na ọ na-ere obere ọnụ ahịa dị mma maka nnukwu mmeri. Na-atụ anya na ọ ga-ejikọta na-arịwanye elu na aghụghọ ndị ọzọ arụmọrụ dị ka FlashAttention, KV-cache quantization, na atụmatụ ọhụrụ dị ka nleba anya nleba anya nke ọtụtụ isi na-agbakọ cache ọbụna n'ihu. Ka windo ndị gbara ya gburugburu na-eto, njikwa nha KV-cache ga-abụ nsogbu imewe etiti, na nkekọrịta ụdị GQA ga-anọgide na-abụ isi ihe.
Mmejuputa n'ezie n'ụwa
Llama 2 70B na Llama 3 na-eji GQA na-eji obere oghere KV jee ozi ogologo oge.
Mbelata ebe nchekwa GPU ka ụdị nkata buru ibu dabara na ngwa ngwa ole na ole ma ọ bụ dị ọnụ ala
Na-eme ka ọgbọ token-site-token dị ngwa na mmepụta API ebe bandwit KV-cache bụ ihe mgbochi
Na-eme ka nha batch buru ibu maka ijere ọtụtụ ndị ọrụ ozi n'otu oge na-enweghị ebe nchekwa na-agwụ ike
Ihe ize ndụ & okporo ụzọ nche
Eziokwu ndị e chepụtara echepụta nwere ike jiri nwayọ tinye akụkọ, nkwado nkwado, ma ọ bụ nsonaazụ nyocha.
Mmetụta ngwa ngwa nwere ike ịmepụta nsonaazụ na-ekwekọghị ekwekọ n'ofe arịrịọ ndị yiri ya.
Enwere ike ikpughe data ederede nwere mmetụta ma ọ bụrụ na njikwa ohere adịghị ike.
Map mmejuputa
Kọwaa usoro mmepụta, ụda, na ụkpụrụ ịdịmma tupu ibugharị.
Weghachite nzaghachi site na isi mmalite ntụkwasị obi mgbe ọ bụla izi ezi dị mkpa.
Debe ebe nleba anya mmadụ maka mpụta dị elu.
Sochie ụkpụrụ ọdịda ma na-azụghachi mkpali ma ọ bụ usoro ọrụ mgbe niile.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Grouped-Query Attention quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Nleba anya ajụjụ ọtụtụ
Ajụjụ a na-ajụkarị
What is Grouped-Query Attention?
Nleba anya ajụjụ ọnụ (GQA) bụ ụzọ isi belata ebe nchekwa achọrọ n'oge ọgbọ ederede site n'ikwe ka ọtụtụ ndị isi ajụjụ kesaa otu igodo na isi uru. Ọ na-eme ka nnukwu ụdị ngwa ngwa na-eje ozi na ọ fọrọ nke nta ka ọ dịghị mma ọnwụ.
Na Nlebanya-ajụjụ ọnụ, kedu ihe a na-ekekọrịta n'etiti otu ndị isi ajụjụ?
GQA na-edobe isi ajụjụ dị iche iche mana na-ahapụ otu ọ bụla n'ime ha kesaa otu isi igodo na isi uru, na-ebelata cache KV.
A kọwara GQA kacha mma dị ka etiti etiti nke oke abụọ?
Multi-isi na-enye onye ọ bụla isi KV nke ya; ọtụtụ ajụjụ na-ekerịta otu KV maka ndị isi niile; GQA na-anọdụ n'etiti ya na otu KV ole na ole.
Kedu akụkụ nke ntinye aka na GQA na-ebute ụzọ dị ọnụ ala?
Nchekwa ahụ na-egosi n'oge ọgbọ, ebe ịgụ cache KV bụ ọnụ ahịa ebe nchekwa-bandwidth kachasị.
Ọ bụrụ na ihe nlereanya nwere isi ajụjụ 32 na otu KV 8, a na-ebelata cache KV site n'ihe dị aṅaa?
Ndị isi ajụjụ 32 kewara site na otu asatọ na-ekekọrịta KV na-enye mbelata ihe dị ka okpukpu anọ na igodo na ụkpụrụ echekwara.
Kedu ihe kpatara GQA nwere ike iji chekwaa ọtụtụ àgwà ihe nlereanya ahụ n'agbanyeghị na-ekerịta isi KV?
Nlebara anya na-anabata igodo na ụkpụrụ nkekọrịta nke ọma karịa ka ọ na-anabata na-efunahụ ntụzịaka ajụjụ dị iche, yabụ GQA na-eme ka ogo dị elu.