GPTQ na AWQ Nyuma yo Guhugura
GPTQ na AWQ nuburyo bubiri buyoboye bwo kugabanya imiterere yindimi zimaze gutozwa kugeza kuri 4-bitike neza kuburyo ikora kubikoresho bihendutse, bito.
Incamake
They are why you can run a capable model on a single consumer GPU instead of a datacenter rack.
Kwibira cyane
Umubare wanyuma wamahugurwa (PTQ) uhagarika moderi yarangiye utongeye kuyitoza, ugashushanya uburemere buringaniye bugera kuri 4 kugirango ugere hafi kimwe cya kane cyo kwibuka. Ikibazo ni ugukora ibi nta gusenya neza. GPTQ (itunganijwe rya OBQ) igereranya ibipimo byapimye kurwego, ikoresheje amakuru ya kabiri yoherejwe kuva dataset ntoya kugirango ihindure uburemere busigaye kandi yishyure buri kosa ryizengurutse. AWQ (Gukora-Kumenya Ibiro Bifite uburemere) ifata indi ngingo: ireba ko agace gato k'imiyoboro yuburemere ari ingenzi cyane, igaragazwa no kureba ubunini bwibikorwa, kandi ikarinda iyo miyoboro yingenzi mugupima aho kuyibara bikabije. Byombi reka moderi nka Llama ikore muri 4-bit, nibikoresho nka vLLM, llama.cpp, na AutoGPTQ byabagize inzira nyamukuru yo gufata ibyemezo byaho kandi bikoresha neza.
Ubushishozi
GPTQ ikoresha igereranya rya Hessian (kugabanuka kw'igihombo) kugirango ihitemo uburyo kuzenguruka ibiro bimwe bigomba guhisha izindi, bikagabanya amakosa yatangijwe. AWQ isimbuka Hessians rwose: ibara ikintu kuri buri muyoboro wapima kugirango imiyoboro yingenzi yuburemere igumane neza neza, hanyuma igereranye kimwe. Byombi bigumane ibikorwa muburyo bunoze kandi bigabanya gusa uburemere, kubera ko uburemere bwiganje mububiko mugihe kugereranya ibikorwa bikunda kubabaza neza.
Ingaruka z'Ingamba
Igiciro na bije
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo bisobanutse
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Kugenzura ubuziranenge
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Kazoza ka GPTQ na AWQ Nyuma yo Guhugura
Quantisation iri gusunika munsi ya 4 bits yerekeza kuri 3-bit, 2-bit, hamwe na gahunda ivanze-yuzuye, akenshi ihujwe nubusa. Tegereza guhuza hamwe na moteri ikora kugirango igabanye, KV-cache compression, hamwe na decoding yibikorwa hamwe. Inkunga yibikoresho kumiterere-bito nka NVFP4 na MXFP4 irakura, kandi ibikoresho byikora bizagenda bitoranya ubugari bwa biti. Intego yagutse ni hafi-gutakaza 4-bit (na hepfo) nkibisanzwe, gukora moderi zikomeye zihendutse gukorera ahantu hose.
Gushyira mu bikorwa Isi
Gukoresha miliyari 70-ibipimo bya Llama kumurongo umwe wa 24 GB ukoresha GPU ukoresheje uburemere bwa 4-bit GPTQ.
Moderi yuzuye ya AWQ yatanzwe kumurongo mwinshi muri vLLM kugirango umusaruro ukorwe neza APIs.
llama.cpp ukoresheje uburemere bwa GGUF kugirango ukoreshe imiterere yururimi kuri mudasobwa igendanwa.
Guhobera Isomero rya AutoGPTQ na AutoAWQ isomero ryemerera abitezimbere kugereranya moderi yakuwe mumirongo mike ya code.
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the GPTQ and AWQ Post-Training Quantization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Ingaruka Imikorere yo Guhugura Ibyatanzwe
Ibibazo bikunze kubazwa
What is GPTQ and AWQ Post-Training Quantization?
GPTQ na AWQ nuburyo bubiri buyobora bwo kugabanya imiterere yindimi zimaze gutozwa kugeza kuri 4-bit neza kuburyo zikoreshwa kubikoresho bihendutse, bito. Ninimpamvu ushobora gukoresha moderi ishoboye kumuguzi umwe GPU aho gukoresha datacenter rack.
Bisobanura iki 'kugereranya nyuma y'amahugurwa' bisobanura?
Umubare wanyuma wamahugurwa ugabanya neza uburemere bwikitegererezo cyarangiye (urugero, kugeza kuri 4 bits) utarinze kubisubiramo.
Ni ayahe makuru GPTQ ikoresha kugirango yishyure amakosa yo kuzenguruka mugihe cyo kubara?
GPTQ ikoresha Hessian igereranya kugirango yumve uburyo kubara ibiro bimwe bigira ingaruka kubihombo, hanyuma igahindura ibiro bisigaye kugirango yishyure.
Ni ubuhe bushishozi bwingenzi butwara AWQ (Gukora-Kumenya Ibiro)?
AWQ igaragaza imiyoboro ihambaye ikoresheje ubunini bwa activation kandi ikabarinda ikoresheje igipimo, kubera ko imiyoboro mike ifite akamaro kanini.
Hafi yangahe yibuka bingana na 4-bit yo kubika hamwe nuburemere bwa 16-bit?
Uhereye kuri 16 bits ukagera kuri 4 kuri buri buremere bigabanya uburemere bwibiro bigera kuri kimwe cya kane, hafi ya 4x kugabanuka.
Ni ukubera iki GPTQ na AWQ byombi bigereranya uburemere ariko bikagumya gukora neza?
Ibiro nibyo byingenzi byibukwa kwibuka, mugihe ibikorwa byunvikana cyane no gutakaza neza, kubwibyo uburemere-gusa nibisanzwe biryoshye.