Tirada
Tiraabidu waxay yaraynaysaa moodalka AI iyadoo lagu kaydinayo nambarada si sax ah hoose, sidaa darteed moodeel u baahan GPU xarun xogeed wuxuu mararka qaarkood ku socon karaa laptop ama taleefan.
Dulmar
It is the main trick that makes large language models cheap and fast enough to deploy widely.
quusid qoto dheer
Shabakadda neerfaha inta badan waa tiro tiro badan oo loo yaqaan miisaanno, sida caadiga ah loo kaydiyo sidii 16- ama 32-bit qiimaha sabbaynaya. Tiro-ururintu waxay dib u kaydisaa miisaannadaas iyadoo la isticmaalayo jajabyo yar, caadi ahaan 8-bit (INT8) ama xitaa 4-bit integers. Ka noqoshada 16-bit ilaa 4-bit waxay dhimaysaa xusuusta qiyaastii afar laab, marka moodeel 70-bilyan ah oo u baahan qiyaastii 140GB at 16-bit wuxuu ku habboonaan karaa qiyaastii 35GB at 4-bit. Nambarada yaryar waxay sidoo kale si degdeg ah ugu socdaan xusuusta, taas oo caadi ahaan dedejisa jiilka. Qabashadu waa saxsanaan: ku tuujin qiyamyo kala duwan oo dhawr heerar ah waxay soo bandhigaysaa qalad wareeg ah. Hababka wanaagsani waxay yareeyaan khasaarahaas iyagoo si taxadar leh u dooranaya qodobbada miisaanka iyo ilaalinta miisaanka ugu xasaasisan, markaa moodeelku wuxuu u dhaqmaa si isku mid ah isagoo isticmaalaya qayb ka mid ah ilaha.
Aragtida Farsamada
Koox kasta oo miisaanno ah waxay helaysaa cunsur miisaan kaas oo khariidado qiyamka dhabta ah ku dhejinaya tiro yar oo miisaan ah; dib ugu dhufashada miisaanku waxay dib u dhisaysaa tiradii asalka ahayd. Hababka qiyaasida tababarka ka dib sida GPTQ iyo AWQ waxay falanqeeyaan xogta xisaabinta yar si ay u go'aamiyaan miisaannada ugu muhiimsan oo deji miisaanka si loo yareeyo khaladka wax soo saarka, halkii ay wax walba si indho la'aan ah u soo koobin lahaayeen. Dhaqdhaqaaqyada waxaa badanaa lagu hayaa saxsanaan sare sababtoo ah waxay ku kala duwan yihiin wakhtiga runtime. Natiijadu waa moodal kaydiya 4-bit istgers laakiin xisaabiya natiijooyinka aad ugu dhow nooca saxda ah.
Saamaynta Istiraatijiyadeed
Xawaaraha iyo miisaanka
Socodka shaqada luqaddu si dhakhso leh ayay u socon kartaa iyada oo aan la hurayn joogteynta.
Helitaanka iyo gaarsiinta
Waxay balaadhisaa gelitaanka luqadaha iyo qaababka isgaarsiinta.
Go'aamo cad
Kooxuhu waxay waqti badan ku qaadan karaan xukunka halka otomaatiggu uu qabanayo ku celcelinta.
Mustaqbalka Tirakoobka
Filo in qiyaasiddu ay noqoto mid aan la soo koobi karin halkii aad ka ahaan lahayd tayayn. Iibiyeyaasha qalabka wax lagu iibiyo waxay ku darayaan 4-bit hooyo iyo xitaa taageero-yar oo hoose, iyo farsamooyinka sida tabobarrada qiyaas-ogaanta ah waxay dubtaan u dulqaadashada saxnaanta hoose ee moodeelka bilawga, yaraynta saxnaanta luminta dheeraad ah. Cilmi baaris lagu sameeyay 2-bit iyo 1-bit (binary) matalaad ayaa firfircoon, ujeedaduna tahay in lagu socodsiiyo moodallo karti u leh taleefannada iyo chips-ka dhejiska ah. Sida qalabka iyo AI gaarka ah u koraan, moodooyinka tayada leh ee tayada leh ayaa udub dhexaad u noqon doona kaaliyeyaasha gudaha iyada oo aan xogta loo dirin daruuraha.
Dhaqangelinta Adduunka-dhabta ah
Ku socodsiinta moodel-sheekeysi sida Llama gudaha GPU-ga macaamiisha iyadoo la adeegsanayo 4-bit GGUF ama GPTQ faylasha halkii aad uga baahnaan lahayd kaarar badan oo xog-xaruneed ah.
Kaaliyeyaasha aaladda ee taleefannada, halkaas oo moodooyinka 8-bit ama 4-bit ay u oggolaadaan astaamaha hadalka iyo qoraalka inay socdaan iyada oo aan lahayn xiriir shabakad.
Jarista kharashyada kala-soocidda daruuraha ee bot-taageerada macmiilka iyadoo u adeegaysa moodal INT8, oo ku habboon codsiyo badan GPU kasta.
Aaladaha cidhifyada sida kamaradaha smart ama dareemayaasha IoT ee ku shaqeeya moodooyinka luuqada aragga ee is haysta oo ku jira xadka xusuusta adag.
Khatarta & Dariiqyada Ilaalada
Xaqiiqooyinka dhalanteed waxay si deggan u geli karaan warbixinnada, taageerada socodka, ama natiijooyinka cilmi-baarista.
Dareenka degdega ahi wuxuu abuuri karaa natiijooyin aan iswaafaqayn codsiyada la midka ah.
Xogta qoraalka xasaasiga ah ayaa laga yaabaa in la kashifo haddii kontaroolada gelitaanka ay daciif yihiin.
Qorshe Hawleedka Dhaqangelinta
Qeex qaabka wax soo saarka, codka, iyo heerarka tayada ka hor inta aan la baahin.
Jawaabaha salka ku haya ilo lagu kalsoon yahay mar kasta oo saxnidu ay muhiim tahay.
Hayso isbaarada dib u eegista bini aadamka ee wax soo saarka sare.
Lasoco qaababka guuldarada oo dib u leyli dardargelinta ama socodka shaqada si joogto ah.
Sii wad Sahaminta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Quantization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Hagaha xiga
Tirinta Vector Haraaga
Su'aalaha soo noqnoqda
What is Quantization?
Tiraabidu waxay yaraynaysaa moodalka AI iyadoo lagu kaydinayo nambarada si sax ah hoose, sidaa darteed moodeel u baahan GPU xarun xogeed wuxuu mararka qaarkood ku socon karaa laptop ama taleefan. Waa khiyaamada ugu weyn ee ka dhigaysa moodooyinka luqadda waa weyn mid raqiis ah oo degdeg ah oo ku filan in si ballaaran loo geeyo.
Waa maxay qiyaasta ugu horraysa ee ku saabsan shabakada neural?
Quantization waxay dib u kaydisaa miisaanka moodeelka saxnaanta tirada hoose, sida 8-bit ama 4-bit integers halkii ay ka ahaan lahayd 16- ama 32-bit sabeyn.
Qiyaastii intee in le'eg xusuusta ayaa lagu keydiyaa iyadoo laga guurayo miisaanka 16-bit ilaa 4-bit?
4-bits waa rubuc ka mid ah 16 bits, markaa kaydinta miisaanku waxay hoos ugu dhacdaa qiyaas ahaan rubuc, qiyaastii 4x dhimis.
Waa maxay dhinaca hoose ee ugu weyn ee tirooyinka gardarrada?
Matalaya qiyamka leh heerar kayar waxay soo bandhigtaa qaladka wareega, kaas oo hoos u dhigi kara tayada wax soo saarka haddii aan si taxadar leh loo maamulin.
Maxay yihiin hababka sida GPTQ iyo AWQ u isticmaalaan kaydinta xogta yar?
Hababkan tababarka kadib waxay falanqeeyaan muunado dhawr ah si loo dejiyo qodobbada miisaanka oo loo ilaaliyo miisaannada xasaasiga ah, iyada oo wax soo saarku ku dhow yihiin asalka.
Waa maxay sababta tiro-ururintu ay inta badan uga dhigayso moodal si dhakhso ah, oo aan ka yarayn?
Qiimaha hoose ee saxda ah wuxuu qaataa xajmiga xajmiga xusuusta yar si uu u raro oo u dhaqaaqo, taas oo inta badan ah cidhiidhiga inta lagu jiro jiilka, sidaas awgeed fikradda ayaa kordhisa.