Ibibazo byinshi
Multi-Ikibazo Cyitonderwa (MQA) nigitekerezo cyo kuzigama kwibuka kugoreka ibitekerezo byahinduwe bihuza urufunguzo rumwe nindangagaciro mumutwe wose.
Incamake
It dramatically speeds up text generation by shrinking the memory the model must shuffle around.
Kwibira cyane
Ibisanzwe-imitwe yitabwaho iha buri mutwe ikibazo cyacyo, urufunguzo, nagaciro kateganijwe. Mugihe cyibisekuru, urufunguzo nindangagaciro kubimenyetso byose byashize bigomba kubikwa no gusubirwamo kuri buri ntambwe - iyi KV cache ihinduka icyuho nyamukuru, kubera ko kuyisoma bivuye mumutwe bitinda kuruta imibare ubwayo. Ibibazo byinshi-byitonderwa, byasabwe na Noam Shazeer muri 2019, bigumya kubaza ibibazo bitandukanye kumutwe ariko bigasenya urufunguzo nagaciro kumutwe umwe usangiwe. Ibi bigabanya cache cache kubintu bingana numubare wumutwe, rimwe na rimwe 8x kugeza 64x nto. Igisubizo kirihuta cyane autoregressive decoding hamwe nibirenge byibuke byibirenge, hamwe nibyiza byoroheje. Hagati yo hagati, Itsinda-Ikibazo Cyitonderwa, iringaniza ubucuruzi-buke.
Ubushishozi
Muri MQA, uburemere bwibibazo buracyatanga H gutandukanya ibibazo bitandukanye, ariko urufunguzo rumwe rukumbi hamwe nigiciro kimwe cyagereranijwe mumitwe yose. Buri mutwe ubara ibitekerezo ukoresheje ikibazo cyarwo urufunguzo rumwe nagaciro. Kuberako kashe ya K na V itagipimwa numubare wumutwe, umurongo wibuke mugihe cya decoding ugabanuka cyane - kandi numuyoboro mugari, ntabwo ubara, nibyo byerekana amarembo yihuta kumuvuduko wihuse.
Ingaruka z'Ingamba
Umuvuduko n'igipimo
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika.
Kugera no kugera
Yagura uburyo bwindimi nuburyo bwo gutumanaho.
Ibyemezo bisobanutse
Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo.
Ejo hazaza h'ibibazo byinshi
MQA yashizeho ko ushobora gukata urufunguzo rwinshi / agaciro imitwe hamwe ningaruka nke, kandi ubushishozi ubu bugira hafi hafi ya LLM yihuta. Umwanya wahujwe ahanini na Grouped-Query Attention (GQA), ikoreshwa muri Llama 2/3 nizindi nyinshi, ikoresha amatsinda make ya KV aho kuba imwe kugirango igarure ubuziranenge mugihe ikomeza umuvuduko mwinshi. Akazi kazoza kahuza ibi bitekerezo na KV-cache compression, kubara, hamwe no kwitondera byinshi-byihuta kugirango bisunikire igihe kirekire kandi serivisi zihendutse.
Gushyira mu bikorwa Isi
Kwihutisha token-by-token generation mubafasha kuganira aho KV cache, ntabwo ari compte mbisi, imipaka yinjira.
__AIU_PROTECTED_11 __ 'PaLM, yakoresheje Multi-Ikibazo Cyitonderwa kugirango ishobore gukora imyanzuro nini nini.
Gukorera abakoresha benshi bahurira kuri GPU imwe mugabanye kubisabwa KV cache yibuka.
Itsinda-Ikibazo Cyitonderwa muri Llama 2 70B na Llama 3, urubyaro rutaziguye ruringaniza umuvuduko wa MQA hamwe nubwiza bwuzuye.
Ingaruka & Kurinda
Ibintu bifatika bishobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi.
Kwihuta byihuse birashobora gukora ibisubizo bidahuye mubisabwa bisa.
Ibyanditswe byumvikana birashobora kugaragara niba kugenzura kugenzura ari ntege.
Igishushanyo mbonera
Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira.
Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro.
Komeza kugenzura abantu kugenzura ibisubizo byinshi.
Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Multi-Query Attention quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Itsinda-Ikibazo
Ibibazo bikunze kubazwa
What is Multi-Query Attention?
Multi-Ikibazo Cyitonderwa (MQA) nigitekerezo cyo kuzigama kwibuka kugoreka ibitekerezo byahinduwe bihuza urufunguzo rumwe nindangagaciro mumutwe wose. Byihuta cyane kubyara inyandiko mugabanya kwibuka moderi igomba guhindagurika.
Niki Multi-Query Attention isangiye mumitwe yose yitabwaho?
MQA ibika ibibazo bitandukanye kumutwe ariko igabana urufunguzo rumwe hamwe nigiciro kimwe cyerekana imitwe yose.
Ni ubuhe butumwa bwibanze MQA ikemura mugihe cya autoregressive generation?
Gusubiramo cache nini ya KV buri ntambwe ni umurongo mugari; MQA igabanya iyo cache kugirango yihutishe decoding.
Hafi yikihe kintu MQA igabanya cache cache?
Kuva urufunguzo nagaciro bisenyuka kuva H imitwe kugeza kuri imwe, cache igabanuka hafi yumutwe.
Ni ubuhe butumwa nyamukuru bwo gukoresha MQA?
Kugabana urufunguzo nagaciro bigabanya ubushobozi bwo guhagararirwa gato, bigatera ireme rito kugabanuka muguhana inyungu nini.
Ni ubuhe buryo bwicaye hagati yimitwe myinshi yitabwaho na MQA?
Itsinda-Ikibazo Cyitonderwa (GQA) rikoresha amatsinda menshi ya KV aho kuba rimwe, iringaniza ubuziranenge n'umuvuduko.