Incamake
Itsinda-Ikibazo Cyitondewe (GQA) nuburyo bwo kugabanya ububiko bukenewe mugihe cyo kubyara inyandiko ureka imitwe myinshi yibibazo isangira urufunguzo rumwe numutwe. Cyakora moderi nini byihuse kugirango ikore hafi nta gihombo cyiza.
Itsinda-Kubaza Icyitonderwa ni igice cyururimi-AI igikoresho cyo gusoma, kubyara, gushyira mu byiciro, no guhindura inyandiko n'imvugo murwego.
Kwibira cyane
Mubisanzwe byinshi-imitwe yitabwaho, buri mutwe ufite ibibazo, urufunguzo, nagaciro. Mugihe cyibisekuru, urufunguzo nagaciro kubintu byose byabanjirije byashizweho ('KV cache') kugirango moderi itabisubiramo. Hamwe nimitwe myinshi hamwe nibice birebire, iyi cache iba nini kandi yiganje mumurongo wibuke mugihe cyagenwe. GQA, yatangijwe nabashakashatsi ba Google mumwaka wa 2023, itondekanya imitwe yibibazo kandi iha buri tsinda umurongo umwe uhuriweho ningenzi kandi byingenzi. Niba ufite imitwe 32 yibibazo ariko amatsinda 8 ya KV gusa, cache cache igabanuka inshuro enye. Ibi bicaye hagati yimitwe myinshi yuzuye (buri mutwe utandukanye) hamwe nibibazo byinshi (umwe asangiye KV kumitwe yose), ifata umuvuduko mwinshi wa MQA mugihe ubuziranenge hafi yubwitonzi bwuzuye. Llama 2 70B na moderi nyinshi nyuma yaje kuyyemeza.
Ubushishozi
Ubwiza bwitondewe buterwa cyane no kugira icyerekezo cyinshi cyibibazo, ariko cyihanganira kugabana urufunguzo nagaciro. GQA ikoresha iyi asimmetrie: igumana imitwe yibibazo byose ariko ikoporora buri mutwe wa KV usangiye kubibazo mumatsinda yayo. Kuzigama biza gufata umwanzuro, aho KV cache niyo nyamukuru ikoresha ububiko bwagutse; imitwe mike ya KV isobanura amakuru make yo gusoma kuri buri kimenyetso cyakozwe. Abanyamideli bakunze 'gutozwa' muri make kugirango bahindure igenzura ryimitwe myinshi ihari muri GQA.
Kumenya Amatsinda-Ikibazo
Itsinda-Ikibazo Cyitondewe (GQA) nuburyo bwo kugabanya ububiko bukenewe mugihe cyo kubyara inyandiko ureka imitwe myinshi yibibazo isangira urufunguzo rumwe numutwe. Cyakora moderi nini byihuse kugirango ikore hafi nta gihombo cyiza. Itsinda-Kubaza Icyitonderwa ni igice cyururimi-AI igikoresho cyo gusoma, kubyara, gushyira mu byiciro, no guhindura inyandiko n'imvugo murwego. Kugirango wubake byimbitse, fata Amatsinda-Kubaza Icyitonderwa nkicyitegererezo gikora, ntabwo ari ikintu kimwe: gusobanura ibyagezweho, gusobanura ibitekerezo, no gutandukanya ibyo sisitemu ishobora gukora byizewe nibisaba guca imanza zinzobere.
Mu myitozo, amakipe akomeye akoresha Itsinda-Ikibazo Cyitonderwa Igishushanyo mbonera, kugarura, no gusubiramo imirongo nka sisitemu imwe y'itumanaho ihuriweho. Bandika ibipimo ngenderwaho byerekana intsinzi, bagerageza kurwanya amakuru afatika hamwe nakazi keza, kandi bagasubiramo bashingiye kubikorwa byagaragaye ko batsinzwe aho gutsinda inshuro imwe. Aha niho imyumvire yubumenyi ihinduka mubushobozi burambye kubicuruzwa, politiki, nibikorwa.
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika. Mugihe kimwe, ibintu bya Hallucinated birashobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi. Uburyo bukomeye cyane ni uguhuza umuvuduko wikigereranyo hamwe na disipuline yimiyoborere: kuyobora abaderevu, gufata ibimenyetso, gutangaza ibyemezo byicyemezo, no gukomeza kuvugurura uburyo bwo kwirinda nkimyitwarire yicyitegererezo, ibyo abakoresha bategereje, nibisabwa n'amategeko bigenda bihinduka.
Ingaruka z'Ingamba
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika.
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika. Mubikorwa byujuje ubuziranenge, ibi bihindurwa mumategeko agenga imikorere, imipaka nyirubwite, hamwe n'imihango yo gusubiramo kenshi kugirango amakipe ashobore kwigirira ikizere aho gupima ibidasobanutse.
Yagura uburyo bwindimi nuburyo bwo gutumanaho.
Yagura uburyo bwindimi nuburyo bwo gutumanaho. Mubikorwa byujuje ubuziranenge, ibi bihindurwa mumategeko agenga imikorere, imipaka nyirubwite, hamwe n'imihango yo gusubiramo kenshi kugirango amakipe ashobore kwigirira ikizere aho gupima ibidasobanutse.
Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo.
Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo. Mubikorwa byujuje ubuziranenge, ibi bihindurwa mumategeko agenga imikorere, imipaka nyirubwite, hamwe n'imihango yo gusubiramo kenshi kugirango amakipe ashobore kwigirira ikizere aho gupima ibidasobanutse.
Gushyira mu bikorwa Isi
Llama 2 70B na Llama 3 ukoresheje GQA kugirango ukorere ibintu birebire hamwe na cache ntoya
Kugabanya ububiko bwa GPU kuburyo moderi nini yo kuganira ihuye na moteri yihuta cyangwa ihendutse
Kwihutisha token-by-token generation mu musaruro APIs aho KV-cache umurongo mugari ari icyuho
Gushoboza ingano nini yo gukorera abakoresha benshi icyarimwe utarinze kwibuka
Uburyo bwo Gushyira mu bikorwa
Itsinda-Ikibazo Cyitonderwa mubikorwa
Llama 2 70B na Llama 3 ukoresheje GQA kugirango ukorere ibintu birebire hamwe na cache ntoya.
Llama 2 70B na Llama 3 bakoresheje GQA kugirango bakore ibintu birebire hamwe na kashe ntoya ya KV Amakipe akunze kubona ibisubizo byiza iyo asobanuye ibipimo byujuje ubuziranenge imbere, agakomeza inzira yo kuzamura abantu kubibazo byimbitse, kandi akurikirana inyungu zibyara umusaruro nibiciro byamakosa mugihe runaka.
Itsinda-Ikibazo Cyitonderwa mubikorwa
Kugabanya ububiko bwa GPU kuburyo moderi nini yo kuganira ihuye na moteri yihuta cyangwa ihendutse.
Kugabanya ububiko bwa GPU kugirango moderi nini yo kuganira ihure na moteri yihuta cyangwa ihendutse Amakipe ubusanzwe abona ibisubizo byiza iyo asobanuye ibipimo ngenderwaho byimbere, agakomeza inzira yo kuzamura abantu kubibazo, kandi akurikirana inyungu zibyara umusaruro hamwe nibiciro byamakosa mugihe.
Itsinda-Ikibazo Cyitonderwa mubikorwa
Kwihutisha token-by-token generation mu musaruro APIs aho KV-cache umurongo mugari ari icyuho.
Kwihutisha ibisekuruza byerekanwa mubikorwa APIs aho KV-cache umurongo mugari ari icyuho Amakipe akunze kubona ibisubizo byiza iyo asobanuye ibipimo ngenderwaho byimbere imbere, agakomeza inzira yo kuzamura abantu kubibazo byimanza, kandi akurikirana inyungu zibyara umusaruro nibiciro byamakosa mugihe runaka.
Itsinda-Ikibazo Cyitonderwa mubikorwa
Gushoboza ingano nini yo gukorera abakoresha benshi icyarimwe utarinze kwibuka.
Gushoboza ingano nini yo gukorera abakoresha benshi icyarimwe nta kunaniza kwibuka Amakipe ubusanzwe abona ibisubizo byiza iyo asobanuye ibipimo ngenderwaho byimbere, agakomeza inzira yo kuzamura abantu kubibazo, kandi akurikirana inyungu zibyara umusaruro hamwe nibiciro byamakosa mugihe.
Ingaruka & Kurinda
Ibintu bifatika bishobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi.
Kwihuta byihuse birashobora gukora ibisubizo bidahuye mubisabwa bisa.
Ibyanditswe byumvikana birashobora kugaragara niba kugenzura kugenzura ari ntege.
Igishushanyo mbonera
Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira.
Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.
Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro.
Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.
Komeza kugenzura abantu kugenzura ibisubizo byinshi.
Komeza kugenzura abantu kugenzura ibisubizo byinshi. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.
Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho.
Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.