Ƙididdigar Samfura
Ƙididdigar ƙirar ƙididdigewa yana rage hanyar sadarwa ta jijiyoyi ta hanyar adana lambobinsa a cikin ƴan ragi, don haka samfurin iri ɗaya yana aiki da sauri kuma akan ƙaramin kayan aiki.
Dubawa
It is the main reason large models can fit on a single GPU, a laptop, or even a phone.
Zurfafa nutsewa
Samfuran da aka horar galibi suna adana kowane nauyi azaman lamba 32-bit ko 16-bit mai iyo. Ƙididdigewa yana maye gurbin waɗanda ke da ƙananan tsari kamar 8-bit integers (INT8) ko 4-bit dabi'u (INT4), yankan ƙwaƙwalwar ajiya kusan 4x zuwa 8x. Samfurin siga-biliyan 70 wanda ke buƙatar kusan 140GB a cikin 16-bit na iya faɗuwa kusa da 35GB a 4-bit, wanda ya dace da GPU mai amfani ɗaya. Kama shine daidaito: matsi da ƙima iri-iri cikin bukiti 256 ko 16 ya rasa dalla-dalla. Hanyoyi na zamani kamar GPTQ, AWQ, da tsarin NF4 da aka yi amfani da su a cikin QLoRA suna ɗaukar abubuwan sikeli mai wayo da kuma kare mafi yawan ma'aunin nauyi, don haka asarar inganci sau da yawa kanana. Ƙididdigewa shine dalilin da ya sa kayan aikin kamar llama.cpp da Ollama zasu iya gudanar da samfura masu inganci a cikin gida ba tare da cibiyar bayanai ba.
Fahimtar Fasaha
Ƙididdige taswirorin ƙididdige ƙididdiga na gaske zuwa ƙaramin grid ɗin lamba ta amfani da sikeli da sifili-point: storage_int = zagaye(darajar / sikelin) + zero_point. Zaɓin ma'auni da kyau shine duka wasan. Sikelin kowane tashoshi ko kowane rukuni yana kiyaye ma'auni daban-daban don yankan matrix nauyi, yana kiyaye daidaiton inda yake da mahimmanci. Ƙididdigar horarwar bayan horo kawai tana canza ƙirar da aka gama, yayin da horarwar ƙididdigewa yana kwatanta zagaye yayin horo don hanyar sadarwar ta koyi jure shi, yawanci tana ba da mafi ƙarancin ƙarancin daidaito.
Dabarun Tasiri
Kudin da kasafin kuɗi
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Shawarwari masu haske
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Kula da inganci
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Makomar Ƙididdigar Samfura
Yi tsammanin daidaici-ƙananan koyaushe ya zama al'ada. Bincike yana tura amintaccen 4-bit, 2-bit, har ma da ma'aunin ma'auni na binary, tare da madaidaitan tsare-tsare waɗanda ke kiyaye yadudduka masu mahimmanci. Hardware yana biye: GPUs da guntuwar waya yanzu sun haɗa da na asali INT8, INT4, da FP8 math units. Siffofin kamar FP8 da MXFP4 suna nufin haɗa kewayon masu iyo tare da girman lamba. Haɗe da fasahohi kamar QLoRA, ƙididdigewa zai ci gaba da yin ƙirar iyaka mai rahusa don aiki da daidaitawa akan na'urorin yau da kullun.
Aiwatar da Gaskiyar Duniya
Gudun samfurin Llama 7B ko 13B akan kwamfutar tafi-da-gidanka tare da llama.cpp ko Ollama ta amfani da fayilolin GGUF 4-bit.
QLoRA yana daidaita babban samfuri akan GPU ɗaya ta hanyar kiyaye ma'aunin tushe a daskare a cikin 4-bit NF4.
Aiwatar da ƙirar INT8 akan wayoyi tare da lokutan aiki akan na'urar don haka mataimaka suna aiki a layi da kuma cikin sirri.
Yin hidimar ƙarshen ƙarshen API mai rahusa inda INT8/FP8 ƙididdigewa kusan ninki biyu kayan aiki da yanke farashin ƙwaƙwalwar ajiya.
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Model Quantization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Rijistar Model
Tambayoyin da ake yawan yi
What is Model Quantization?
Ƙididdigar ƙirar ƙididdigewa yana rage hanyar sadarwa ta jijiyoyi ta hanyar adana lambobinsa a cikin ƴan ragi, don haka samfurin iri ɗaya yana aiki da sauri kuma akan ƙaramin kayan aiki. Wannan shine babban dalilin da yasa manyan samfura zasu iya dacewa akan GPU guda ɗaya, kwamfutar tafi-da-gidanka, ko ma waya.
Menene ƙididdige ƙirar ƙididdigewa da farko game da hanyar sadarwar jijiya?
Ƙididdigar ƙididdigewa tana adana sigogi iri ɗaya a cikin ƙananan raƙuman ruwa (misali INT8 ko INT4 maimakon 16- ko 32-bit floats), rage ƙwaƙwalwar ajiya da saurin lissafi.
Kusan nawa ƙwaƙwalwar ajiya zata iya canza samfurin daga 16-bit zuwa 4-bit ajiyewa?
Tafi daga 16 rago a kowane nauyi zuwa 4 ragowa yana yanke ajiya da kusan juzu'i huɗu, wanda shine dalilin da yasa manyan samfura zasu iya dacewa da GPU guda ɗaya.
Menene babban abin da ke haifar da ƙididdige ƙididdigewa mai ƙanƙanci?
Yin taswirar ƙima iri-iri zuwa ƴan matakan ƙima yana rasa daki-daki, wanda zai iya rage inganci sai dai in ƙwanƙwasa mai wayo yana kare ma'aunin nauyi.
Ta yaya horon sanin ƙididdiga ya bambanta da ƙididdigewa bayan horo?
Horowar ƙididdigewa yana gina kuskuren kewayawa cikin madauki na horo, don haka hanyar sadarwa ta daidaita kuma yawanci tana kiyaye ƙarin daidaito a ƙananan ƙananan nisa.
Wace dabara ce ke amfani da ƙididdigewa na 4-bit don yin kyawawan samfura masu araha akan GPU ɗaya?
QLoRA yana kiyaye ƙirar tushe a daskare a cikin tsarin 4-bit NF4 kuma yana horar da ƙananan ma'aunin adaftar, yana barin manyan samfuran su kasance masu kyau a daidaita su akan kayan aiki masu sauƙi.