Itsinda-Ikibazo
Itsinda-Ikibazo Cyitondewe (GQA) nuburyo bwo kugabanya ububiko bukenewe mugihe cyo kubyara inyandiko ureka imitwe myinshi yibibazo isangira urufunguzo rumwe numutwe.
Incamake
It makes large models much faster to serve with almost no quality loss.
Kwibira cyane
Mubisanzwe byinshi-imitwe yitabwaho, buri mutwe ufite ibibazo, urufunguzo, nagaciro. Mugihe cyibisekuru, urufunguzo nagaciro kubintu byose byabanjirije byashizweho ('KV cache') kugirango moderi itabisubiramo. Hamwe nimitwe myinshi hamwe nibice birebire, iyi cache iba nini kandi yiganje mumurongo wibuke mugihe cyagenwe. GQA, yatangijwe nabashakashatsi ba Google mumwaka wa 2023, itondekanya imitwe yibibazo kandi iha buri tsinda umurongo umwe uhuriweho ningenzi kandi byingenzi. Niba ufite imitwe 32 yibibazo ariko amatsinda 8 ya KV gusa, cache cache igabanuka inshuro enye. Ibi bicaye hagati yimitwe myinshi yuzuye (buri mutwe utandukanye) hamwe nibibazo byinshi (umwe asangiye KV kumitwe yose), ifata umuvuduko mwinshi wa MQA mugihe ubuziranenge hafi yubwitonzi bwuzuye. Llama 2 70B na moderi nyinshi nyuma yaje kuyyemeza.
Ubushishozi
Ubwiza bwitondewe buterwa cyane no kugira icyerekezo cyinshi cyibibazo, ariko cyihanganira kugabana urufunguzo nagaciro. GQA ikoresha iyi asimmetrie: igumana imitwe yibibazo byose ariko ikoporora buri mutwe wa KV usangiye kubibazo mumatsinda yayo. Kuzigama biza gufata umwanzuro, aho KV cache niyo nyamukuru ikoresha ububiko bwagutse; imitwe mike ya KV isobanura amakuru make yo gusoma kuri buri kimenyetso cyakozwe. Abanyamideli bakunze 'gutozwa' muri make kugirango bahindure igenzura ryimitwe myinshi ihari muri GQA.
Ingaruka z'Ingamba
Umuvuduko n'igipimo
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika.
Kugera no kugera
Yagura uburyo bwindimi nuburyo bwo gutumanaho.
Ibyemezo bisobanutse
Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo.
Ejo hazaza h'itsinda-Ikibazo
GQA ubu isanzwe isanzwe muburyo bwo gufungura uburemere kuko igurisha neza igiciro gito cyiza kubitsinzi binini. Witegereze guhuza byinshi hamwe nubundi buryo bukoreshwa neza nka FlashAtention, KV-cache igereranya, hamwe na gahunda nshya nkimitwe myinshi yihishe yitonze ikanda cache ndetse ikarenza. Mugihe imiterere ya Windows ikura, kugenzura ingano ya KV-cache bizagumaho ikibazo cyo gushushanya hagati, kandi GQA-uburyo bwo kugabana umutwe bizakomeza kuba urufunguzo.
Gushyira mu bikorwa Isi
Llama 2 70B na Llama 3 ukoresheje GQA kugirango ukorere ibintu birebire hamwe na cache ntoya
Kugabanya ububiko bwa GPU kuburyo moderi nini yo kuganira ihuye na moteri yihuta cyangwa ihendutse
Kwihutisha token-by-token generation mu musaruro APIs aho KV-cache umurongo mugari ari icyuho
Gushoboza ingano nini yo gukorera abakoresha benshi icyarimwe utarinze kwibuka
Ingaruka & Kurinda
Ibintu bifatika bishobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi.
Kwihuta byihuse birashobora gukora ibisubizo bidahuye mubisabwa bisa.
Ibyanditswe byumvikana birashobora kugaragara niba kugenzura kugenzura ari ntege.
Igishushanyo mbonera
Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira.
Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro.
Komeza kugenzura abantu kugenzura ibisubizo byinshi.
Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Grouped-Query Attention quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Ibibazo byinshi
Ibibazo bikunze kubazwa
What is Grouped-Query Attention?
Itsinda-Ikibazo Cyitondewe (GQA) nuburyo bwo kugabanya ububiko bukenewe mugihe cyo kubyara inyandiko ureka imitwe myinshi yibibazo isangira urufunguzo rumwe numutwe. Cyakora moderi nini byihuse kugirango ikore hafi nta gihombo cyiza.
Mumatsinda-Ikibazo Cyitonderwa, niki gisangiwe mumatsinda yibibazo?
GQA ibika imitwe yibibazo bitandukanye ariko ikareka buri tsinda ryabo bagabana urufunguzo rumwe rwimitwe nagaciro, kugabanya Kache cache.
GQA isobanurwa neza nkimpamvu yo hagati hagati yizo ntambwe ebyiri?
Multi-head itanga buri mutwe wacyo KV; ibibazo byinshi bisangiye KV imwe kumitwe yose; GQA yicaye hagati hamwe nitsinda rito rya KV.
Ni ikihe gice cyo gufata umwanzuro GQA ikora cyane cyane?
Kuzigama bigaragarira mugihe cyibisekuru, aho gusoma KV cache aribiganza kwibuka-umurongo mugari.
Niba icyitegererezo gifite imitwe 32 yibibazo hamwe nitsinda 8 KV, ubwihisho bwa KV bugabanuka nikihe kintu?
32 ibibazo byibibazo bigabanijwe nitsinda 8 risangiwe KV ritanga hafi kugabanuka inshuro enye murufunguzo rwihishwa nagaciro.
Kuki GQA ishobora kubika ubwiza bwikitegererezo nubwo igabana imitwe ya KV?
Icyitonderwa cyihanganira kugabana urufunguzo nagaciro keza kuruta kwihanganira gutakaza icyerekezo cyibibazo bitandukanye, GQA rero ikomeza ubuziranenge hejuru.