Ƙididdigewa
Ƙididdige ƙididdigewa yana rage ƙirar AI ta hanyar adana lambobinsa a ƙananan madaidaici, don haka samfurin da ke buƙatar GPU na cibiyar bayanai na iya aiki a wani lokaci akan kwamfutar tafi-da-gidanka ko waya.
Dubawa
It is the main trick that makes large language models cheap and fast enough to deploy widely.
Zurfafa nutsewa
Cibiyar sadarwa ta jijiyoyi galibi babban tarin lambobi ne da ake kira ma'auni, yawanci ana adana su azaman ƙimar 16- ko 32-bit masu iyo. Ƙididdigewa yana sake adana waɗannan ma'aunin ta amfani da ƙananan ragi, yawanci 8-bit (INT8) ko ma ma'aunin 4-bit. Tafi daga 16-bit zuwa 4-bit yana yanke ƙwaƙwalwar ajiya kusan ninki huɗu, don haka ƙirar siga-biliyan 70 da ke buƙatar kusan 140GB a 16-bit na iya dacewa da kusan 35GB akan 4-bit. Ƙananan lambobi kuma suna motsawa cikin sauri ta hanyar ƙwaƙwalwar ajiya, wanda yawanci yana haɓaka tsarawa. Kamun shine daidaito: matsi ɗimbin dabi'u cikin ƴan matakai yana gabatar da kuskuren zagaye. Hanyoyi masu kyau suna rage wannan asara ta hanyar zabar abubuwan ƙima a hankali da kuma kare mafi yawan ma'aunin nauyi, don haka ƙirar tana nuna kusan iri ɗaya yayin amfani da ɗan yanki na albarkatun.
Fahimtar Fasaha
Kowane rukuni na ma'aunin nauyi yana samun ma'aunin ma'auni wanda ke yin taswirar ƙididdiga na gaske akan ƙaramin adadin adadin; ninka baya da ma'auni kusan yana sake gina ainihin lambar. Hanyoyin ƙididdige horo bayan horo kamar GPTQ da AWQ suna nazarin ƙaramin saitin daidaitawa don yanke shawarar wane ma'auni mafi mahimmanci da saita ma'auni don rage kuskuren fitarwa, maimakon zagaye komai a makance. Sau da yawa ana kiyaye kunnawa a mafi girman daidaito saboda sun bambanta sosai a lokacin aiki. Sakamakon shine samfurin da ke adana intigers 4-bit amma yana ƙididdige sakamako kusa da cikakkiyar sigar.
Dabarun Tasiri
Gudu da sikelin
Gudun aikin harshe na iya tafiya da sauri ba tare da sadaukar da daidaito ba.
Shiga ku isa
Yana faɗaɗa damar shiga cikin harsuna da salon sadarwa.
Shawarwari masu haske
Ƙungiyoyi za su iya ciyar da ƙarin lokaci akan hukunci yayin da aiki da kai ke sarrafa maimaitawa.
Makomar Quantization
Yi tsammanin ƙididdigewa ya zama tsoho maimakon ingantawa. Masu siyar da kayan aikin suna ƙara ɗan ƙasa na 4-bit har ma da tallafin ƙananan-bit, da dabaru kamar juriya-sanin horo ga yin gasa don ƙarancin daidaito cikin ƙirar daga farkon, rage ƙimar daidaito gaba. Bincike cikin wakilcin 2-bit da 1-bit (binary) yana aiki, yana nufin gudanar da samfura masu inganci akan wayoyi da kwakwalwan kwamfuta da aka saka. Kamar yadda na'ura da masu zaman kansu AI ke girma, ingantattun ƙididdiga masu ƙididdigewa za su kasance tsakiya ga masu taimakawa a cikin gida ba tare da aika bayanai zuwa gajimare ba.
Aiwatar da Gaskiyar Duniya
Gudanar da ƙirar taɗi kamar Llama a cikin gida akan GPU mai amfani ta amfani da fayilolin GGUF 4-bit ko GPTQ maimakon buƙatar katunan cibiyar bayanai da yawa.
Mataimakan kan na'urar akan wayoyi, inda nau'ikan 8-bit ko 4-bit ke barin fasalin magana da rubutu suyi aiki ba tare da haɗin yanar gizo ba.
Yanke ƙimar ƙimar girgije don bot na goyon bayan abokin ciniki ta hanyar bautar da ƙirar INT8, dacewa da ƙarin buƙatun akan kowane GPU.
Na'urorin Edge kamar kyamarori masu wayo ko na'urori masu auna firikwensin IoT suna gudanar da ƙayyadaddun ƙirar harsunan hangen nesa a cikin iyakokin ƙwaƙwalwar ajiya.
Hatsari & Tsare-tsare
Abubuwan da aka ruɗe suna iya shigar da rahotanni cikin nutsuwa, kwararar tallafi, ko abubuwan bincike.
Hankali na gaggawa na iya ƙirƙirar sakamako mara daidaituwa a cikin buƙatun iri ɗaya.
Za a iya fallasa bayanan rubutu mai ma'ana idan ikon samun dama yana da rauni.
Taswirar Hanya
Ƙayyade tsarin fitarwa, sautin, da ma'auni masu inganci kafin fitowa.
Amsa a ƙasa tare da amintattun tushe a duk lokacin da daidaito ya shafi mahimmanci.
Ajiye wurin binciken ɗan adam don abubuwan da ake samu masu girma.
Bibiyar tsarin gazawar kuma sake horar da tsokaci ko tafiyar aiki akai-akai.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Quantization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Ragowar Vector Quantization
Tambayoyin da ake yawan yi
What is Quantization?
Ƙididdige ƙididdigewa yana rage ƙirar AI ta hanyar adana lambobinsa a ƙananan madaidaici, don haka samfurin da ke buƙatar GPU na cibiyar bayanai na iya aiki a wani lokaci akan kwamfutar tafi-da-gidanka ko waya. Ita ce babbar dabarar da ke sa manyan nau'ikan harshe arha da saurin isa don turawa ko'ina.
Menene ƙididdigewa da farko ke canzawa game da hanyar sadarwa na jijiyoyi?
Ƙididdige ƙididdigewa yana sake adana ma'aunin ƙirar a ƙananan daidaitattun lambobi, kamar 8-bit ko 4-bit integers maimakon 16- ko 32-bit na iyo.
Kusan nawa ne aka adana ƙwaƙwalwar ajiya ta hanyar motsa ma'auni daga 16-bit zuwa 4-bit?
4 bits shine kashi ɗaya cikin huɗu na ragi 16, don haka ajiyar nauyi yana raguwa zuwa kusan kwata, kusan raguwar 4x.
Menene babban fa'ida na ƙididdige ƙididdigewa?
Ma'anar ƙima tare da ƙananan matakan yana gabatar da kuskuren zagaye, wanda zai iya lalata ingancin fitarwa idan ba a sarrafa shi a hankali ba.
Menene hanyoyin kamar GPTQ da AWQ suke amfani da ƙaramin saitin bayanai don?
Waɗannan hanyoyin horarwa suna nazarin ƴan abubuwan shigar da samfur don saita abubuwan ƙima da kare ma'aunin nauyi, adana abubuwan da aka fitar kusa da asali.
Me yasa ƙididdigewa sau da yawa ke yin samfurin sauri, ba ƙarami ba?
Ƙididdiga masu ƙanƙanta suna ɗaukar ƙarancin bandwidth na ƙwaƙwalwar ajiya don ɗaukarwa da motsawa, wanda galibi shine ƙwanƙwasa yayin tsarawa, don haka ƙima yana sauri.