Ukunakwa Kwemibuzo Ehlanganisiwe
I-Grouped-Query Attention (GQA) iyindlela yokunciphisa inkumbulo edingekayo phakathi nokukhiqizwa kombhalo ngokuvumela izinhloko zemibuzo ezimbalwa zabelane ngokhiye ofanayo namakhanda enani.
Uhlolojikelele
It makes large models much faster to serve with almost no quality loss.
I-Deep Dive
Kusendlalelo sokunaka esinamakhanda amaningi, inhloko ngayinye inemibuzo yayo, okhiye, namanani. Ngesikhathi sokukhiqiza, okhiye namanani awo wonke amathokheni adlule afakwe kunqolobane ('inqolobane ye-KV') ukuze imodeli ingawabuyiseli. Ngamakhanda amaningi nezimo ezinde, le cache iba yinkulu futhi ibusa umkhawulokudonsa wememori ngesikhathi sokunquma. I-GQA, eyethulwe Google abacwaningi ngo-2023, ihlanganisa izinhloko zemibuzo futhi inikeze iqembu ngalinye isethi eyodwa eyabelwe yokhiye namakhanda yenani. Uma unamakhanda emibuzo angu-32 kodwa amaqembu angu-8 KV kuphela, inqolobane ye-KV incipha cishe ngokuphindwe kane. Lokhu kuhlala phakathi kokunaka okugcwele kwamakhanda amaningi (ikhanda ngalinye lihlukene) kanye nokunakwa kwemibuzo eminingi (i-KV eyodwa eyabelwe kuwo wonke amakhanda), ithwebula iningi lejubane le-MQA kuyilapho igcina ikhwalithi iseduze nokunakwa okugcwele. I-Llama 2 70B kanye namamodeli amaningi akamuva ayamukela.
I-Technical Insight
Ikhwalithi yokunaka incike kakhulu ekubeni nezikhombisi-ndlela eziningi ezihlukene zemibuzo, kodwa ibekezelela ukwabelana ngokhiye namanani. I-GQA isebenzisa le-asymmetry: igcina yonke imibuzo kodwa iphindaphinda ikhanda ngalinye le-KV elabelwe kuyo yonke imibuzo eseqenjini layo. Ukongiwa kuza ngokulinganisa, lapho inqolobane ye-KV ingumthengi oyinhloko womkhawulokudonsa wenkumbulo; amakhanda e-KV ambalwa asho idatha encane okufanele ifundwe ngethokheni ngayinye ekhiqiziwe. Amamodeli avame 'ukuqeqeshwa' kafushane ukuze aguqule indawo yokuhlola enamakhanda amaningi abe yi-GQA.
I-Strategic Impact
Isivinini nesikali
Ukugeleza komsebenzi wolimi kungahamba ngokushesha ngaphandle kokudela ukuvumelana.
Finyelela futhi ufinyelele
Yandisa ukufinyelela kuzo zonke izilimi nezitayela zokuxhumana.
Izinqumo ezicacile
Amaqembu angachitha isikhathi esiningi ekwahluleleni kuyilapho i-automation isingatha impinda.
Ikusasa Lokunakwa Kwemibuzo Ehlanganisiwe
I-GQA manje isiwukuzenzakalelayo okujwayelekile kumamodeli anesisindo esivulekile ngoba ihweba ngokuhlanzekile ngenani elincane lekhwalithi yokuwina okukhulu. Lindela ukuthi ihlangane ngokuqhubekayo namanye amaqhinga asebenza kahle afana ne-FlashAttention, i-KV-cache quantization, nezikimu ezintsha ezifana nokunakwa okucashile kwamakhanda amaningi acindezela inqolobane nakakhulu. Njengoba amawindi womongo ekhula, ukulawula usayizi wenqolobane ye-KV kuzohlala kuyinkinga yokuklama emaphakathi, futhi ukwabelana kwekhanda kwesitayela se-GQA kuzohlala kuyisici esibalulekile.
Ukuqaliswa Komhlaba Wangempela
I-Llama 2 70B kanye ne-Llama 3 isebenzisa i-GQA ukuze inikeze okuqukethwe okude ngenqolobane encane ye-KV
Kwehliswa inkumbulo ye-GPU ukuze imodeli yengxoxo enkulu ilingane kuma-accelerator ambalwa noma ashibhile
Ukusheshisa ukukhiqizwa kwamathokheni ngamathokheni kuma-API okukhiqiza lapho umkhawulokudonsa we-KV-cache uyibhodlela
Inika amandla osayizi benqwaba abakhulu bokusebenzela abasebenzisi abaningi ngesikhathi esisodwa ngaphandle kokuqeda inkumbulo
Izingozi & Guardrails
Amaqiniso akhonjiwe angafaka ngokuthula imibiko, ukugeleza kosekelo, noma imiphumela yocwaningo.
Ukuzwela okusheshayo kungadala imiphumela engahambisani kuzo zonke izicelo ezifanayo.
Idatha yombhalo ebucayi ingase idalulwe uma izilawuli zokufinyelela zibuthakathaka.
Ukuqalisa Umhlahlandlela
Chaza ifomethi yokuphumayo, ithoni, namazinga wekhwalithi ngaphambi kokukhishwa.
Izimpendulo eziyisisekelo ngemithombo ethembekile noma nini lapho ukunemba kubalulekile.
Gcina indawo yokuhlola isibuyekezo somuntu ukuze uthole imiphumela ephezulu.
Landela amaphethini okuhluleka futhi uqeqeshe kabusha imiyalo noma ukuhamba komsebenzi njalo.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Grouped-Query Attention quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Umhlahlandlela olandelayo
Ukunakwa kwemibuzo eminingi
Imibuzo evame ukubuzwa
What is Grouped-Query Attention?
I-Grouped-Query Attention (GQA) iyindlela yokunciphisa inkumbulo edingekayo phakathi nokukhiqizwa kombhalo ngokuvumela izinhloko zemibuzo ezimbalwa zabelane ngokhiye ofanayo namakhanda enani. Kwenza amamodeli amakhulu asheshe kakhulu ukuze asebenze cishe ngaphandle kokulahlekelwa kwekhwalithi.
Ekunakeni Kombuzo Ohlanganisiwe, yini okwabelwana ngayo phakathi kweqembu lezinhloko zemibuzo?
I-GQA igcina amakhanda emibuzo ehlukene kodwa ivumela iqembu ngalinye ukuthi labelane ngesethi eyodwa yokhiye namakhanda wenani, inciphisa inqolobane ye-KV.
I-GQA ichazwa kangcono njengendawo ephakathi nendawo phakathi kwaziphi izimo ezimbili ezeqisayo?
I-Multi-head inikeza inhloko ngayinye i-KV yayo; imibuzo eminingi yabelana nge-KV eyodwa kuwo wonke amakhanda; I-GQA ihlezi phakathi namaqembu ambalwa e-KV.
Iyiphi ingxenye ye-GQA eyenza ishibhile?
Ukongiwa kuvela ngesikhathi sokukhiqiza, lapho ukufunda inqolobane ye-KV kuyizindleko ezihamba phambili ze-memory-bandwidth.
Uma imodeli inamakhanda emibuzo angu-32 namaqembu angu-8 KV, inqolobane ye-KV yehliswa cishe ngayiphi into?
Amakhanda emibuzo angu-32 ahlukaniswe ngamaqembu angu-8 e-KV okwabelwana ngawo anikeza cishe ukuncishiswa okuphindwe kane kokhiye namanani afakwe kunqolobane.
Kungani i-GQA ingagcina iningi lekhwalithi yemodeli naphezu kokwabelana ngamakhanda e-KV?
Ukunaka kubekezelela okhiye bokwabelana namanani kangcono kakhulu kunokubekezelela ukulahlekelwa izikhombisi-ndlela ezihlukile zemibuzo, ngakho-ke i-GQA igcina ikhwalithi iphezulu.