GPTQ da AWQ Ƙididdigar Koyarwa Bayan Koyarwa
GPTQ da AWQ hanyoyi ne na jagoranci guda biyu don raguwar ƙirar harshe da aka riga aka horar zuwa daidaitattun 4-bit don haka suna gudana akan rahusa, ƙaramin kayan aiki.
Dubawa
They are why you can run a capable model on a single consumer GPU instead of a datacenter rack.
Zurfafa nutsewa
Ƙididdigar horon bayan horo (PTQ) yana matsawa samfurin da aka gama ba tare da sake horar da shi ba, yana tsara ma'aunin ma'auni mai tsayi zuwa ragi 4 zuwa kusan kwata ƙwaƙwalwar ajiya. Kalubalen shine yin hakan ba tare da lalata daidaito ba. GPTQ (gyaran OBQ) yana ƙididdige ma'aunin ma'auni ta Layer, ta amfani da bayanin oda na biyu daga ƙaramin bayanan daidaitawa don daidaita sauran ma'aunin nauyi da rama kowane kuskuren zagaye. AWQ (Ayyukan Ƙimar Ma'aunin Kunnawa) yana ɗaukar wani kusurwa daban: yana lura cewa ƙananan tashoshi masu nauyi ba su da mahimmanci, an gano su ta hanyar kallon girman kunnawa, kuma yana kare waɗannan tashoshi masu mahimmanci ta hanyar ƙididdige su da ƙarfi. Dukansu suna barin samfura kamar Llama su gudana a cikin 4-bit, kuma kayan aikin kamar vLLM, llama.cpp, da AutoGPTQ sun sanya su na yau da kullun don ƙimar gida da ingantaccen farashi.
Fahimtar Fasaha
GPTQ yana amfani da kimanin Hessian (curvature of the loss) don yanke shawarar yadda zagayawa ɗaya nauyi ya kamata ya motsa sauran, rage girman kuskuren da aka gabatar. AWQ ya tsallake Hessians gabaɗaya: yana ƙididdige ma'aunin sikelin kowane tashoshi ta yadda mahimman tashoshi masu nauyi su kiyaye ingantaccen ingancin su, sannan ƙididdige su daidai. Dukansu suna ci gaba da kunnawa cikin madaidaicin madaidaici kuma kawai damfara ma'aunin nauyi, tunda ma'aunin nauyi ya mamaye ƙwaƙwalwar ajiya yayin da ƙididdige ƙididdigewa yana ƙoƙarin cutar da daidaito sosai.
Dabarun Tasiri
Kudin da kasafin kuɗi
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Shawarwari masu haske
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Kula da inganci
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Makomar GPTQ da AWQ Ƙididdigar Koyarwa Bayan Koyarwa
Ƙididdigar ƙididdigewa tana turawa ƙasa 4 ragowa zuwa 3-bit, 2-bit, da tsare-tsare-daidaitacce, yawanci haɗe da ɓatanci. Yi tsammanin haɗin kai kusa da injunan hidima don haka ƙididdigewa, matsewar KV-cache, da ƙididdige ƙima suna aiki tare. Tallafin kayan masarufi don tsarin ƙananan-bit kamar NVFP4 da MXFP4 yana girma, kuma kayan aikin sarrafa kansa za su ƙara ɗaukar faɗuwar bit-layer. Babban burin yana kusa-rasa 4-bit (kuma ƙasa) azaman tsoho, yana yin samfura masu ƙarfi mai arha don hidima a ko'ina.
Aiwatar da Gaskiyar Duniya
Gudun samfurin Llama na siga na biliyan 70 akan GPU mabukaci guda 24 GB ta amfani da ma'aunin GPTQ 4-bit.
Samfura masu ƙididdigewa na AWQ waɗanda aka yi aiki a babban kayan aiki a cikin vLLM don APIs samarwa masu inganci.
llama.cpp ta amfani da ma'aunin GGUF mai ƙididdigewa don gudanar da ƙirar harshe a cikin gida akan kwamfutar tafi-da-gidanka CPU.
Rungumar Face's AutoGPTQ da ɗakunan karatu na AutoAWQ yana barin masu haɓaka ƙididdige samfurin da aka zazzage cikin ƴan layukan lamba.
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the GPTQ and AWQ Post-Training Quantization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Tasirin Ayyuka don Batun Bayanan Horarwa
Tambayoyin da ake yawan yi
What is GPTQ and AWQ Post-Training Quantization?
GPTQ da AWQ hanyoyi ne na jagoranci guda biyu don raguwar ƙirar harshe da aka riga aka horar zuwa daidaitattun 4-bit don haka suna gudana akan rahusa, ƙaramin kayan aiki. Su ne dalilin da ya sa za ku iya gudanar da ingantaccen samfuri akan GPU mabukaci guda ɗaya maimakon taragon datacenter.
Menene ma'anar 'ƙididdigar bayan horo'?
Ƙididdigar horo bayan horo yana rage madaidaicin ma'aunin ƙirar da aka gama (misali, zuwa rago 4) ba tare da sake horar da shi daga karce ba.
Wane bayani GPTQ ke amfani da shi don rama kurakurai yayin ƙididdigewa?
GPTQ yana amfani da madaidaicin Hessian don fahimtar yadda ƙididdige nauyi ɗaya ke shafar asarar, sannan kuma daidaita sauran ma'aunin nauyi don ramawa.
Wane mahimmin mahimmin fahimta ne ke tafiyar da AWQ (Kididdigar Nauyin Kunnawa)?
AWQ yana gano tashoshi masu nauyi ta amfani da girman kunnawa kuma yana kare su ta hanyar sikeli, tunda ƴan tashoshi suna da alaƙa da rashin daidaituwa.
Kusan nawa ƙwaƙwalwar ajiyar 4-bit ke adanawa tare da ma'aunin 16-bit?
Tafi daga 16 ragowa zuwa 4 rago a kowane nauyi yana yanke ƙwaƙwalwar nauyi zuwa kusan kwata, kusan raguwar 4x.
Me yasa duka GPTQ da AWQ suke yawan ƙididdige ma'auni amma suna ci gaba da kunnawa a daidaici mafi girma?
Nauyi shine babban farashin ƙwaƙwalwar ajiya, yayin da kunnawa sun fi kula da asara daidai, don haka ƙididdige nauyi-kawai shine wuri mai daɗin gama gari.