Nleba anya ajụjụ ọtụtụ
Multi-Query Attention (MQA) bụ ntụgharị nchekwa nchekwa na nleba anya nke mgbanwe na-ekerịta otu igodo na ụkpụrụ n'ofe isi nlebara anya niile.
Nchịkọta
It dramatically speeds up text generation by shrinking the memory the model must shuffle around.
Ime miri emi
Nlebara anya ọtụtụ isi ọkọlọtọ na-enye onye isi ọ bụla ajụjụ ajụjụ, igodo ya na amụma uru ya. N'oge ọgbọ, igodo na ụkpụrụ maka akara ngosi niile gara aga ga-echekwa ma bugharịa ya na nzọụkwụ ọ bụla - cache KV a na-aghọ ihe bụ isi, ebe ọ bụ na ịgụ ya site na ebe nchekwa dị nwayọọ karịa mgbakọ na mwepụ n'onwe ya. Ntị ajụjụ ọtụtụ, nke Noam Shazeer tụpụtara na 2019, na-edobe amụma ajụjụ dị iche iche n'otu isi mana ọ daa igodo na ụkpụrụ n'otu isi na-ekekọrịta. Nke a na-ebelata cache KV site n'ihe ruru ọnụ ọgụgụ isi, mgbe ụfọdụ 8x ruo 64x pere mpe. Ihe si na ya pụta bụ ngbanwe akpaaka ngwa ngwa yana akara ukwu ebe nchekwa dị nfe, na-enwe naanị njiri mara mma. Ebe etiti, Nleba anya ajụjụ ọnụ, na-eme ka azụmaahịa ahụ guzozie.
Nghọta nka nka
Na MQA, arọ ajụjụ ka na-emepụta vector ajụjụ H dị iche, mana otu ntule isi na ntule otu uru na-ekekọrịta n'ofe isi niile. Onye isi ọ bụla na-agbakọ nlebara anya na-eji ajụjụ nke ya megide otu igodo na ụkpụrụ. N'ihi na echere K na V tenors anaghịzi enwe ọnụ ọgụgụ nke isi, bandwidth ebe nchekwa n'oge ngbanwe na-ada nke ọma - yana bandwit, ọ bụghị ịgbakọ, bụ ihe ọnụ ụzọ ámá na-agba ọsọ na ngwa ngwa ọgbara ọhụrụ.
Mmetụta atụmatụ
Ọsọ na ọnụ ọgụgụ
Usoro ọrụ asụsụ nwere ike ịga ngwa ngwa n'achụghị nkwụsi ike.
Nweta na iru
Ọ na-agbasawanye ohere n'ofe asụsụ na ụdị nzikọrịta ozi.
Mkpebi doro anya
Ndị otu nwere ike itinyekwu oge na ikpe ebe akpaaka na-ejikwa nkwughachi.
Ọdịnihu nke nlebara anya ọtụtụ ajụjụ
MQA kwadoro na ị nwere ike ihichapụ isi igodo/uru bara uru na-enweghị mmerụ ahụ, na nghọta ahụ na-akpụzi ihe fọrọ nke nta ka ọ bụrụ ngwa ngwa ọ bụla LLM. Ogige ahụ agbakọtala nke ukwuu na Grouped-Query Attention (GQA), ejiri na Llama 2/3 na ọtụtụ ndị ọzọ, nke na-eji otu KV ole na ole karịa otu iji nwetaghachi ogo ma na-edobe ọtụtụ n'ime ngwa ngwa. Ọrụ ga-eme n'ọdịnihu na-agwakọta echiche ndị a na mkpakọ KV-cache, quantization, na multi-latent nlebara anya iji kwalite ọnọdụ dị ogologo na ozi dị ọnụ ala.
Mmejuputa n'ezie n'ụwa
Na-eme ka ọgbọ token site-token ọsọ ọsọ na ndị enyemaka nkata ebe nchekwa KV, ọ bụghị mkpokọta raw, na-amachi ntinye.
Google's PaLM, nke jiri Multi-Query Attention mee ka ntinye aka buru ibu rụọ ọrụ nke ọma.
Ijere ọtụtụ ndị ọrụ na-emekọ ihe ọnụ n'otu GPU site na ibelata ebe nchekwa nchekwa KV nke ọ bụla.
Nlebanya ajụjụ ọnụ nke agbakọtara na Llama 2 70B na Llama 3, nwa nwa na-edozi ọsọ MQA na nlebara anya zuru oke.
Ihe ize ndụ & okporo ụzọ nche
Eziokwu ndị e chepụtara echepụta nwere ike jiri nwayọ tinye akụkọ, nkwado nkwado, ma ọ bụ nsonaazụ nyocha.
Mmetụta ngwa ngwa nwere ike ịmepụta nsonaazụ na-ekwekọghị ekwekọ n'ofe arịrịọ ndị yiri ya.
Enwere ike ikpughe data ederede nwere mmetụta ma ọ bụrụ na njikwa ohere adịghị ike.
Map mmejuputa
Kọwaa usoro mmepụta, ụda, na ụkpụrụ ịdịmma tupu ibugharị.
Weghachite nzaghachi site na isi mmalite ntụkwasị obi mgbe ọ bụla izi ezi dị mkpa.
Debe ebe nleba anya mmadụ maka mpụta dị elu.
Sochie ụkpụrụ ọdịda ma na-azụghachi mkpali ma ọ bụ usoro ọrụ mgbe niile.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Multi-Query Attention quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Nlebanya Ajụjụ agbakọtara
Ajụjụ a na-ajụkarị
What is Multi-Query Attention?
Multi-Query Attention (MQA) bụ ntụgharị nchekwa nchekwa na nleba anya nke mgbanwe na-ekerịta otu igodo na ụkpụrụ n'ofe isi nlebara anya niile. Ọ na-eme ka ọgbọ ederede dị ngwa ngwa site na ibelata ebe nchekwa ihe nlereanya ahụ ga-agbagharị gburugburu.
Gịnị ka Multi-ajụjụ Ntị na-ekekọrịta n'ofe ndị isi nlebara anya niile?
MQA na-edobe ajụjụ dị iche iche n'otu isi mana na-ekerịta otu ntule isi na otu ntụle uru n'ofe isi niile.
Kedu ihe bụ isi okwu MQA na-ekwu n'oge ọgbọ na-agba ọsọ?
Ịbugharị nnukwu oghere KV nke ọ bụla na-ejikọta bandwidth; MQA na-ebelata cache ahụ iji mee ka ngbanwe dị ngwa.
Site n'ụzọ dị aṅaa ka MQA na-ebelata cache KV?
Ebe igodo na ụkpụrụ dara site na isi H gaa na otu, cache na-agbada site na mkpokọta isi.
Kedu ihe bụ isi azụmaahịa nke iji MQA?
Ịkekọrịta igodo na ụkpụrụ na-ebelata ikike nnọchite anya ntakịrị, na-eme ka mbelata àgwà dị nta na mgbanwe maka nnukwu ọsọ ọsọ.
Kedu ọdịiche dị n'etiti nlebara anya ọtụtụ isi na MQA?
Grouped-Query Attention (GQA) na-eji ọtụtụ KV otu kama otu, na-edozi ogo na ọsọ.