I-GPTQ kanye ne-AWQ Post-Training Quantization
I-GPTQ ne-AWQ izindlela ezimbili ezihamba phambili zokunciphisa amamodeli olimi asevele aqeqeshelwe ukuya ekunembeni okuyi-4-bit ukuze asebenzise ihadiwe eshibhile, encane.
Uhlolojikelele
They are why you can run a capable model on a single consumer GPU instead of a datacenter rack.
I-Deep Dive
I-Post-training quantization (PTQ) icindezela imodeli eqediwe ngaphandle kokuyiqeqesha kabusha, yenza imephu izisindo ezinembe kakhulu zehle ziye kumabhithi angu-4 ukuze cishe ikota yenkumbulo. Inselele ukwenza lokhu ngaphandle kokuphazamisa ukunemba. I-GPTQ (ukuthuthukiswa kwe-OBQ) ilinganisa ungqimba lwesisindo ngongqimba, isebenzisa ulwazi lwe-oda lesibili olusuka kudathasethi encane yokulinganisa ukuze kulungiswe izisindo ezisele futhi kunxeshezelwe iphutha ngalinye lokuqoqa. I-AWQ (I-Activation-aware Weight Quantization) ithatha i-engeli ehlukile: ibona ukuthi ingxenye encane yamashaneli esisindo ibaluleke ngokulinganayo, ikhonjwa ngokubheka ubukhulu bokwenza kusebenze, futhi ivikela lawo mashaneli abalulekile ngokukala kunokuwalinganisa ngamandla. Womabili avumela amamodeli afana ne-Llama ukuthi asebenze ku-4-bit, futhi amathuluzi afana ne-vLLM, i-llama.cpp, ne-AutoGPTQ awenze ajwayelekile ekuqondeni kwasendaweni nokonga imali.
I-Technical Insight
I-GPTQ isebenzisa ukulinganisa kwe-Hessian (ijika lokulahlekelwa) ukuze inqume ukuthi ukuzungeza isisindo esisodwa kufanele kugudluze kanjani ezinye, kuncishiswe iphutha elethuliwe. I-AWQ yeqa ama-Hessians ngokuphelele: ibala isici sokukala sesiteshi ngasinye ukuze iziteshi ezibalulekile zesisindo zigcine ukunemba kwazo okusebenzayo, bese zilinganisa ngokulinganayo. Kokubili kugcina ukwenza kusebenze ngokunemba okuphezulu futhi cindezela izisindo kuphela, njengoba izisindo zibusa inkumbulo kuyilapho ukwenza kusebenze ukulinganisa kuvame ukulimaza ukunemba okwengeziwe.
I-Strategic Impact
Izindleko kanye nesabelomali
Izinqumo zezakhiwo ziqhuba ukusebenza kanye nezindleko zokusebenza iminyaka.
Izinqumo ezicacile
Imfundo yobuchwepheshe isiza amaqembu ukuthi akhethe isitaki esifanele, hhayi nje esisha.
Ukulawulwa kwekhwalithi
Izinketho ezingcono zobunjiniyela zinciphisa izehlakalo ezinokwethenjelwa ekukhiqizeni.
Ikusasa le-GPTQ kanye ne-AWQ Post-Training Quantization
I-Quantization iphusha ngaphansi kwamabhithi angu-4 kuya ku-3-bit, 2-bit, kanye nezikimu ezinembile ezixubile, ngokuvamile ezihlanganiswe nobuncane. Lindela ukusondelana okusondelene nezinjini ezinikezayo ukuze ukwandise, ukucindezelwa kwenqolobane ye-KV, kanye nokuqopha okuqagelayo kusebenza ndawonye. Ukusekelwa kwezingxenyekazi zekhompuyutha zamafomethi ebhithi ephansi njenge-NVFP4 ne-MXFP4 kuyakhula, futhi amathuluzi azenzakalelayo azokhetha ngokuqhubekayo ububanzi bebhithi yesendlalelo ngasinye. Umgomo obanzi ucishe ulahlekelwe ngu-4-bit (nangaphansi) njengokuzenzakalelayo, okwenza amamodeli aqinile ashibhe ukuze asebenze yonke indawo.
Ukuqaliswa Komhlaba Wangempela
Isebenzisa imodeli ye-Llama yepharamitha engu-70-bhiliyoni ku-GPU eyodwa yomthengi ongu-24 GB isebenzisa izisindo ze-GPTQ ezingu-4-bit.
Amamodeli anenani le-AWQ asetshenziswa ekuphumeni okuphezulu ku-vLLM kuma-API okukhiqiza akongayo.
I-llama.cpp isebenzisa izisindo ze-GGUF ezilinganiselwe ukusebenzisa amamodeli olimi endaweni kukhompuyutha ephathekayo ye-CPU.
Imitapo yolwazi ye-Hugging Face's AutoGPTQ kanye ne-AutoAWQ evumela onjiniyela balinganisele imodeli elandiwe ngemigqa embalwa yekhodi.
Izingozi & Guardrails
Ukuthuthukisa ibhentshimakhi eyodwa kungafihla ubuthakathaka obubanzi besistimu.
Izindleko zengqalasizinda nezokulungisa zivame ukubukelwa phansi.
Izikhala zokuphepha nokubonakala zingakhula njengoba izinhlelo ziba nzima kakhulu.
Ukuqalisa Umhlahlandlela
Chaza ukubambezeleka, ikhwalithi, nezindleko ezihlosiwe ngaphambi kokuqaliswa.
Ibhentshimakhi ngaphansi komthwalo wangempela nezimo zedatha.
Ukuqapha amathuluzi amaphutha, ukukhukhuleka, nomthelela wabasebenzisi.
Lungiselela izindlela zokuhlehlisa nezigameko ngaphambi kokukala.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the GPTQ and AWQ Post-Training Quantization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Umhlahlandlela olandelayo
Ithonya Imisebenzi Yokubalulwa Kwedatha Yokuqeqesha
Imibuzo evame ukubuzwa
What is GPTQ and AWQ Post-Training Quantization?
I-GPTQ ne-AWQ izindlela ezimbili ezihamba phambili zokunciphisa amamodeli olimi asevele aqeqeshelwe ukuya ekunembeni okuyi-4-bit ukuze asebenzise ihadiwe eshibhile, encane. Kungakho ungasebenzisa imodeli enekhono ku-GPU yomthengi oyedwa esikhundleni se-datacenter rack.
Kusho ukuthini 'i-post-training quantization'?
Ukulinganisa kwangemuva kokuqeqeshwa kunciphisa ukunemba kwezisindo zemodeli eqediwe (isb., ukuya kumabhithi angu-4) ngaphandle kokuyiqeqesha kabusha kusukela ekuqaleni.
Yiluphi ulwazi olusetshenziswa i-GPTQ ukunxephezela amaphutha okuqoqa lapho kulinganisa?
I-GPTQ isebenzisa i-Hessian elinganiselwe ukuze iqonde ukuthi ukulinganisa isisindo esisodwa kukuthinta kanjani ukulahlekelwa, bese ilungisa izisindo ezisele ukuze inxephezele.
Yikuphi ukuqonda okubalulekile okushayela i-AWQ (I-Activation-aware Weight Quantization)?
I-AWQ ihlonza amashaneli esisindo abalulekile isebenzisa ubukhulu bokwenza kusebenze futhi iwavikele ngokukala, njengoba iziteshi ezimbalwa zibalulekile ngokungafani.
Cishe ingakanani inkumbulo esindisa i-4-bit quantization uma iqhathaniswa nezisindo eziyi-16-bit?
Ukusuka kumabhithi angu-16 kuye kwangu-4 ngesisindo ngasinye kunciphisa inkumbulo yesisindo ibe cishe ingxenye yesine, cishe ukuncishiswa okungu-4x.
Kungani kokubili i-GPTQ ne-AWQ kuvamise ukukala izisindo kodwa kugcine ukwenza kusebenze ngokunemba okuphezulu?
Izisindo ziyizindleko zenkumbulo eziyinhloko, kuyilapho ukwenza kusebenze kuzwela kakhulu ekulahlekelweni okunembayo, ngakho ukulinganisa isisindo kuphela kuyindawo evamile yobumnandi.