Hankalin Tambaya da yawa
Multi-Query Attention (MQA) juzu'i ce ta adana ƙwaƙwalwar ajiya akan hankalin taswira wanda ke raba saiti ɗaya na maɓalli da ƙima a duk shugabannin hankali.
Dubawa
It dramatically speeds up text generation by shrinking the memory the model must shuffle around.
Zurfafa nutsewa
Daidaitaccen kulawar kai mai yawa yana bawa kowane shugaban tambayar kansa, maɓalli, da hasashen ƙimarsa. A lokacin tsarawa, maɓallai da ƙimar duk alamun da suka gabata dole ne a adana su kuma a sake ɗora su a kowane mataki - wannan cache na KV ya zama babban ƙugiya, tunda karanta shi daga ƙwaƙwalwar ajiya yana da hankali fiye da lissafin kansa. Hankalin Tambayoyi da yawa, wanda Noam Shazeer ya gabatar a cikin 2019, yana adana tsinkaye daban-daban akan kowane kai amma yana ruguza maɓallai da ƙimar zuwa kai guda ɗaya. Wannan yana rage ma'ajin KV da ma'auni daidai da adadin kawunan, wani lokacin 8x zuwa 64x ƙarami. Sakamakon yana da saurin yankewa autoregressive da saurin sawun ƙwaƙwalwar ajiya, tare da tsoma mai inganci kawai. Ƙasa ta tsakiya, Ƙungiya-Tambaya Hankali, tana daidaita cinikin.
Fahimtar Fasaha
A cikin MQA, ma'aunin tambaya har yanzu yana samar da nau'ikan tambayoyin H daban-daban, amma tsinkayar maɓalli ɗaya da tsinkayar ƙima guda ɗaya ana raba su a duk kawunansu. Kowane shugaban yana lissafta hankali ta amfani da tambayarsa akan maɓalli da ƙima iri ɗaya. Saboda abubuwan da aka adana K da V tenors ba su ƙara yin sikelin da adadin kawunan ba, bandwidth na ƙwaƙwalwar ajiya yayin yanke hukunci yana faɗuwa sosai - kuma bandwidth, ba ƙididdigewa ba, shine saurin haɓakar ƙofofin kan masu haɓakawa na zamani.
Dabarun Tasiri
Gudu da sikelin
Gudun aikin harshe na iya tafiya da sauri ba tare da sadaukar da daidaito ba.
Shiga ku isa
Yana faɗaɗa damar shiga cikin harsuna da salon sadarwa.
Shawarwari masu haske
Ƙungiyoyi za su iya ciyar da ƙarin lokaci akan hukunci yayin da aiki da kai ke sarrafa maimaitawa.
Makomar Hankalin Tambaya da yawa
MQA ta kafa cewa zaku iya datse kawunan maɓalli / ƙima ba tare da lahani kaɗan ba, kuma wannan fahimtar yanzu yana siffata kusan kowane saurin saurin LLM. Filin ya haɗu da yawa akan Hannun Tambayar Rukuni (GQA), wanda aka yi amfani da shi a Llama 2/3 da sauran su, waɗanda ke amfani da ƴan rukunin KV maimakon ɗaya don dawo da inganci yayin kiyaye mafi yawan saurin gudu. Aiki na gaba yana haɗa waɗannan ra'ayoyin tare da matsawa KV-cache, ƙididdigewa, da kulawar ɓoyayyiya da yawa don tura dogon yanayi da sabis mai rahusa.
Aiwatar da Gaskiyar Duniya
Ƙaddamar da ƙayyadaddun ƙayyadaddun alama a cikin mataimakan taɗi inda KV cache, ba daɗaɗɗen ƙididdigewa ba, yana iyakance kayan aiki.
Google's PaLM, wanda yayi amfani da Hankalin Tambayoyi da yawa don ba da damar ingantacciyar fa'ida mai girma.
Bauta wa masu amfani da yawa na lokaci guda akan GPU ɗaya ta hanyar rage ƙwaƙwalwar ajiyar cache na KV-kowace bukata.
Hankalin Rukuni-Tambaya a Llama 2 70B da Llama 3, zuriyar kai tsaye tana daidaita saurin MQA tare da cikakkiyar kulawa.
Hatsari & Tsare-tsare
Abubuwan da aka ruɗe suna iya shigar da rahotanni cikin nutsuwa, kwararar tallafi, ko abubuwan bincike.
Hankali na gaggawa na iya ƙirƙirar sakamako mara daidaituwa a cikin buƙatun iri ɗaya.
Za a iya fallasa bayanan rubutu mai ma'ana idan ikon samun dama yana da rauni.
Taswirar Hanya
Ƙayyade tsarin fitarwa, sautin, da ma'auni masu inganci kafin fitowa.
Amsa a ƙasa tare da amintattun tushe a duk lokacin da daidaito ya shafi mahimmanci.
Ajiye wurin binciken ɗan adam don abubuwan da ake samu masu girma.
Bibiyar tsarin gazawar kuma sake horar da tsokaci ko tafiyar aiki akai-akai.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Multi-Query Attention quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Hankalin Tambaya-Rukuni
Tambayoyin da ake yawan yi
What is Multi-Query Attention?
Multi-Query Attention (MQA) juzu'i ce ta adana ƙwaƙwalwar ajiya akan hankalin taswira wanda ke raba saiti ɗaya na maɓalli da ƙima a duk shugabannin hankali. Yana hanzarta haɓaka rubutun rubutu ta hanyar raguwar ƙwaƙwalwar ajiyar ƙirar dole ne ta shuɗe.
Menene Hankalin Tambayoyi da yawa ke rabawa a duk shugabannin hankali?
MQA yana adana tambayoyi daban-daban akan kowane kai amma yana raba hasashe maɓalli ɗaya da tsinkayar ƙima guda ɗaya a duk shugabannin.
Menene babban ƙulli na farko da MQA ke magancewa yayin tsarar da ba ta dace ba?
Sake loda babban cache na KV kowane mataki yana da ɗaure bandwidth; MQA yana raguwa wannan cache don hanzarta yanke hukunci.
Ta wajen wanne dalili MQA ke rage cache na KV?
Tun da maɓallai da ƙima suna rugujewa daga shugabannin H zuwa ɗaya, cache ɗin yana raguwa da kusan ƙidayar kai.
Menene babban ciniki na amfani da MQA?
Raba maɓallai da ƙima yana rage ƙarfin wakilci kaɗan, yana haifar da raguwar ƙarancin inganci a musanya don babban riba mai sauri.
Wanne bambance-bambance ne ke zaune tsakanin daidaitaccen kulawar kai da yawa da MQA?
Ƙungiya-Query Attention (GQA) yana amfani da ƙungiyoyin KV da yawa maimakon ɗaya, daidaita inganci da sauri.