HejuruUbuyobozi bukurikira
Ingaruka Imikorere yo Guhugura Ibyatanzwe
Tekiniki
Ubuyobozi bwa tekiniki
GPTQ na AWQ nuburyo bubiri buyoboye bwo kugabanya imiterere yindimi zimaze gutozwa kugeza kuri 4-bitike neza kuburyo ikora kubikoresho bihendutse, bito.
Umva iki gitaboAI-byavuzwe · ~2 min
Ninimpamvu ushobora gukoresha moderi ishoboye kumuguzi umwe GPU aho kuba datacenter rack.
Umubare wanyuma wamahugurwa (PTQ) uhagarika moderi yarangiye utongeye kuyitoza, ugashushanya uburemere buringaniye bugera kuri 4 kugirango ugere hafi kimwe cya kane cyo kwibuka. Ikibazo ni ugukora ibi nta gusenya neza. GPTQ (itunganijwe rya OBQ) igereranya ibipimo byapimye kurwego, ikoresheje amakuru ya kabiri yoherejwe kuva dataset ntoya kugirango ihindure uburemere busigaye kandi yishyure buri kosa ryizengurutse. AWQ (Gukora-Kumenya Ibiro Bifite uburemere) ifata indi ngingo: ireba ko agace gato k'imiyoboro yuburemere ari ingenzi cyane, igaragazwa no kureba ubunini bwibikorwa, kandi ikarinda iyo miyoboro yingenzi mugupima aho kuyibara bikabije. Byombi reka moderi nka Llama ikore muri 4-bit, nibikoresho nka vLLM, llama.cpp, na AutoGPTQ byabagize inzira nyamukuru yo gufata ibyemezo byaho kandi bikoresha neza.
Gukoresha miliyari 70-ibipimo bya Llama kumurongo umwe wa 24 GB ukoresha GPU ukoresheje uburemere bwa 4-bit GPTQ.
Moderi yuzuye ya AWQ yatanzwe kumurongo mwinshi muri vLLM kugirango umusaruro ukorwe neza APIs.
llama.cpp ukoresheje uburemere bwa GGUF kugirango ukoreshe imiterere yururimi kuri mudasobwa igendanwa.
Guhobera Isomero rya AutoGPTQ na AutoAWQ isomero ryemerera abitezimbere kugereranya moderi yakuwe mumirongo mike ya code.
Quantisation iri gusunika munsi ya 4 bits yerekeza kuri 3-bit, 2-bit, hamwe na gahunda ivanze-yuzuye, akenshi ihujwe nubusa. Tegereza guhuza hamwe na moteri ikora kugirango igabanye, KV-cache compression, hamwe na decoding yibikorwa hamwe. Inkunga yibikoresho kumiterere-bito nka NVFP4 na MXFP4 irakura, kandi ibikoresho byikora bizagenda bitoranya ubugari bwa biti. Intego yagutse ni hafi-gutakaza 4-bit (na hepfo) nkibisanzwe, gukora moderi zikomeye zihendutse gukorera ahantu hose.
Ibaruwa y'amakuru y'ubuntu
Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.
Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.
Isuzume
Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.
Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano
GPTQ na AWQ nuburyo bubiri buyobora bwo kugabanya imiterere yindimi zimaze gutozwa kugeza kuri 4-bit neza kuburyo zikoreshwa kubikoresho bihendutse, bito. Ninimpamvu ushobora gukoresha moderi ishoboye kumuguzi umwe GPU aho gukoresha datacenter rack.
Umubare wanyuma wamahugurwa ugabanya neza uburemere bwikitegererezo cyarangiye (urugero, kugeza kuri 4 bits) utarinze kubisubiramo.
GPTQ ikoresha Hessian igereranya kugirango yumve uburyo kubara ibiro bimwe bigira ingaruka kubihombo, hanyuma igahindura ibiro bisigaye kugirango yishyure.
AWQ igaragaza imiyoboro ihambaye ikoresheje ubunini bwa activation kandi ikabarinda ikoresheje igipimo, kubera ko imiyoboro mike ifite akamaro kanini.
Uhereye kuri 16 bits ukagera kuri 4 kuri buri buremere bigabanya uburemere bwibiro bigera kuri kimwe cya kane, hafi ya 4x kugabanuka.
Ibiro nibyo byingenzi byibukwa kwibuka, mugihe ibikorwa byunvikana cyane no gutakaza neza, kubwibyo uburemere-gusa nibisanzwe biryoshye.
Komeza wige
Abandi bayobozi batoranijwe kuriyi ngingo
HejuruUbuyobozi bukurikira
Ingaruka Imikorere yo Guhugura Ibyatanzwe
Tekiniki