Ukunakwa kwemibuzo eminingi
I-Multi-Query Attention (MQA) iyi-twist egcina inkumbulo ekunakeni kwe-transformer eyabelana ngesethi eyodwa yokhiye kanye namanani kuwo wonke amakhanda okunaka.
Uhlolojikelele
It dramatically speeds up text generation by shrinking the memory the model must shuffle around.
I-Deep Dive
Ukunaka okujwayelekile kwamakhanda amaningi kunikeza inhloko ngayinye umbuzo wayo, ukhiye, kanye nokuqagela kwenani. Ngesikhathi sokukhiqiza, okhiye namanani awo wonke amathokheni adlule kufanele afakwe kunqolobane futhi alayishwe kabusha esinyathelweni ngasinye - le cache ye-KV iba ibhodlela eliyinhloko, njengoba ukuyifunda ngekhanda kuhamba kancane kunezibalo ngokwazo. I-Multi-Query Attention, ehlongozwe nguNoam Shazeer ngo-2019, igcina imibuzo ehlukene ngekhanda ngalinye kodwa igoqa okhiye namanani ekhanda elilodwa okwabelwana ngalo. Lokhu kunciphisa inqolobane ye-KV ngesici esilingana nenani lamakhanda, ngezinye izikhathi 8x kuya ku-64x amancane. Umphumela uba ukuqoshwa kwe-autoregressive okusheshayo kanye nenkumbulo elula, enekhwalithi ephansi yediphu. Indawo emaphakathi, Ukunakwa Kombuzo Ohlanganisiwe, kulinganisa ukuhwebelana.
I-Technical Insight
Ku-MQA, izisindo zombuzo zisakhiqiza amavekhtha wemibuzo angu-H ahlukene, kodwa ukuqagela kokhiye owodwa kanye nokuqagela kwenani elilodwa kwabelwana ngawo wonke amakhanda. Ikhanda ngalinye libala ukunaka lisebenzisa owalo umbuzo ngokumelene nokhiye namanani afanayo. Ngenxa yokuthi izithambisi ezigcinwe kunqolobane ze-K kanye ne-V azisalingani nenani lamakhanda, umkhawulokudonsa wenkumbulo ngesikhathi sokukhishwa kwekhodi wehla kakhulu - futhi umkhawulokudonsa, hhayi wokubala, yilokho isivinini sokukhiqiza amasango kuma-accelerator esimanje.
I-Strategic Impact
Isivinini nesikali
Ukugeleza komsebenzi wolimi kungahamba ngokushesha ngaphandle kokudela ukuvumelana.
Finyelela futhi ufinyelele
Yandisa ukufinyelela kuzo zonke izilimi nezitayela zokuxhumana.
Izinqumo ezicacile
Amaqembu angachitha isikhathi esiningi ekwahluleleni kuyilapho i-automation isingatha impinda.
Ikusasa Lokunakwa Kwemibuzo Eminingi
I-MQA ithole ukuthi ungakwazi ukuthena amakhanda okhiye/inani angasadingeki ngokulimala okuncane, futhi lokho kuqonda manje sekubumba cishe yonke i-LLM esheshayo. Inkambu ihlangane kakhulu ku-Grouped-Query Attention (GQA), esetshenziswa ku-Llama 2/3 nezinye eziningi, esebenzisa amaqembu ambalwa e-KV kunelilodwa ukuze kubuyiselwe ikhwalithi ngenkathi kugcinwa iningi lokusheshisa. Umsebenzi wesikhathi esizayo uhlanganisa le mibono nokucindezelwa kwenqolobane ye-KV, ukulinganisa, nokunaka okufihlekile okuningi ukuze kuphushwe izimo ezinde kanye nokuphakelwa okushibhile.
Ukuqaliswa Komhlaba Wangempela
Ukusheshisa ukukhiqizwa kwamathokheni kubasizi bengxoxo lapho inqolobane ye-KV, hhayi ikhompuyutha eluhlaza, ikhawulela ukuphuma.
Google's PaLM, esebenzise i-Multi-Query Attention ukuze inike amandla ukucabangela ngezinga elikhulu.
Ukukhonza abasebenzisi abaningi ngesikhathi esisodwa ku-GPU eyodwa ngokunciphisa inkumbulo yenqolobane ye-KV yesicelo ngasinye.
Ukunakwa Kwemibuzo Ehlanganisiwe ku-Llama 2 70B kanye ne-Llama 3, inzalo eqondile elinganisa isivinini se-MQA nekhwalithi yokunaka okugcwele.
Izingozi & Guardrails
Amaqiniso akhonjiwe angafaka ngokuthula imibiko, ukugeleza kosekelo, noma imiphumela yocwaningo.
Ukuzwela okusheshayo kungadala imiphumela engahambisani kuzo zonke izicelo ezifanayo.
Idatha yombhalo ebucayi ingase idalulwe uma izilawuli zokufinyelela zibuthakathaka.
Ukuqalisa Umhlahlandlela
Chaza ifomethi yokuphumayo, ithoni, namazinga wekhwalithi ngaphambi kokukhishwa.
Izimpendulo eziyisisekelo ngemithombo ethembekile noma nini lapho ukunemba kubalulekile.
Gcina indawo yokuhlola isibuyekezo somuntu ukuze uthole imiphumela ephezulu.
Landela amaphethini okuhluleka futhi uqeqeshe kabusha imiyalo noma ukuhamba komsebenzi njalo.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Multi-Query Attention quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Umhlahlandlela olandelayo
Ukunakwa Kwemibuzo Ehlanganisiwe
Imibuzo evame ukubuzwa
What is Multi-Query Attention?
I-Multi-Query Attention (MQA) iyi-twist egcina inkumbulo ekunakeni kwe-transformer eyabelana ngesethi eyodwa yokhiye kanye namanani kuwo wonke amakhanda okunaka. Isheshisa ukukhiqizwa kombhalo ngokunciphisa inkumbulo okufanele imodeli ishove nxazonke.
I-Multi-Query Attention yabelana ngani kuzo zonke izinhloko zokunakwa?
I-MQA igcina imibuzo ehlukene ngekhanda ngalinye kodwa yabelana ngokuqagela kokhiye owodwa kanye nokuqagela kwenani elilodwa kuwo wonke amakhanda.
Iyini ibhodlela eliyinhloko i-MQA elibhekana nayo ngesikhathi sokukhiqiza ngokuzenzakalelayo?
Ukulayisha kabusha inqolobane enkulu ye-KV isinyathelo ngasinye siboshelwe kumkhawulokudonsa; I-MQA incipha leyo cache ukuze kusheshiswe ukuqoshwa kwekhodi.
Cishe iyiphi into eyenza i-MQA inciphise inqolobane ye-KV?
Njengoba okhiye namanani ehla esuka kumakhanda e-H ukuya kwelilodwa, inqolobane iyancipha cishe ngokubala kwekhanda.
Ikuphi ukuhwebelana okukhulu kokusebenzisa i-MQA?
Ukwabelana ngokhiye namanani kunciphisa umthamo wokumelela kancane, okubangela ukwehla kwekhwalithi encane ekuhwebeni kwezinzuzo ezinkulu zesivinini.
Yikuphi okuhlukile okuhlala phakathi kokunaka okujwayelekile kwamakhanda amaningi kanye ne-MQA?
I-Grouped-Query Attention (GQA) isebenzisa amaqembu amaningana e-KV esikhundleni selilodwa, ibhalansisa ikhwalithi nesivinini.