Umubare w'icyitegererezo
Kwigereranya kwicyitegererezo bigabanya urusobe rwimitsi ibika umubare wabyo muke, bityo moderi imwe ikora byihuse no kubikoresho bito.
Incamake
It is the main reason large models can fit on a single GPU, a laptop, or even a phone.
Kwibira cyane
Moderi yatojwe mubisanzwe ibika buri buremere nka 32-bit cyangwa 16-bito ireremba-ingingo. Quantisation isimbuza abafite imiterere-yuzuye yo hasi nka 8-bit integer (INT8) cyangwa 4-biti (INT4), guca kwibuka hafi 4x kugeza 8x. Moderi ya miliyari 70-ikeneye hafi 140GB muri 16-bit irashobora kugabanuka hafi ya 35GB kuri 4-bit, bikwiranye na GPU imwe. Gufata ni ukuri: gukanda indangagaciro zitandukanye mu ndobo 256 cyangwa 16 zitakaza ibisobanuro. Uburyo bugezweho nka GPTQ, AWQ, nuburyo bwa NF4 bukoreshwa muri QLoRA butoranya ibintu bipima ubwenge kandi bikarinda uburemere bworoshye, bityo gutakaza ubuziranenge akenshi ni bito. Quantisation niyo mpamvu ibikoresho nka llama.cpp na Ollama bishobora gukoresha moderi zishoboye mugace nta data center.
Ubushishozi
Ikarita yerekana agaciro nyako kuri gride ntoya ikoresheje umunzani na zeru-point: yabitswe_int = uruziga (agaciro / igipimo) + zero_ point. Guhitamo igipimo neza ni umukino wose. Kuri buri muyoboro cyangwa kuri buri tsinda gupima umunzani utandukanye kubice bya matrix yuburemere, bikabika neza aho bifite akamaro. Umubare wanyuma wamahugurwa uhindura gusa icyitegererezo cyarangiye, mugihe imyitozo-yo kumenya kugereranya kwigana mugihe cyamahugurwa kugirango urusobe rwige kubyihanganira, mubisanzwe bitanga neza-bito byukuri.
Ingaruka z'Ingamba
Igiciro na bije
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo bisobanutse
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Kugenzura ubuziranenge
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Ejo hazaza h'icyitegererezo
Tegereza burigihe-hasi neza kugirango ibe ibisanzwe. Ubushakashatsi burimo gusunika kwizerwa 4-bit, 2-bit, ndetse nuburemere bwa binary, wongeyeho gahunda ivanze-itomoye ituma ibyiyumvo byoroshye biri hejuru. Ibyuma birakurikira: GPUs na chip ya terefone ubu harimo INT8 kavukire, INT4, na FP8. Imiterere nka FP8 na MXFP4 igamije guhuza intera ireremba hamwe nubunini bwa integer. Ufatanije nubuhanga nka QLoRA, kubara bizakomeza gutuma imipaka yerekana imipaka ihendutse gukora no guhuza neza ibikoresho bya buri munsi.
Gushyira mu bikorwa Isi
Gukoresha moderi ya 7B cyangwa 13B Llama kuri mudasobwa igendanwa hamwe na llama.cpp cyangwa Ollama ukoresheje dosiye ya 4-bit ya GGUF.
QLoRA gutunganya neza moderi nini kuri GPU imwe ukomeza uburemere bwibanze bukonjeshwa muri 4-bit NF4.
Kohereza moderi ya INT8 kuri terefone hamwe nigihe cyo gukora-ibikoresho kugirango abafasha bakore kumurongo kandi wenyine.
Gukora bihendutse API iheruka aho INT8 / FP8 igereranya ikubye kabiri ibyinjira kandi igabanya ikiguzi cyo kwibuka.
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Model Quantization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Kwiyandikisha by'icyitegererezo
Ibibazo bikunze kubazwa
What is Model Quantization?
Kwigereranya kwicyitegererezo bigabanya urusobe rwimitsi ibika umubare wabyo muke, bityo moderi imwe ikora byihuse no kubikoresho bito. Ninimpamvu nyamukuru moderi nini zishobora guhura na GPU imwe, mudasobwa igendanwa, cyangwa na terefone.
Ni ubuhe buryo bwo kugereranya icyitegererezo buhinduka cyane cyane kubyerekeye urusobe rw'imitsi?
Quantization ibika ibipimo bimwe mubice bike (urugero INT8 cyangwa INT4 aho kureremba 16- cyangwa 32-bit), kugabanya kwibuka no kwihutisha imibare.
Hafi ya bangahe yibuka ishobora guhindura moderi kuva 16-bit kugeza kuri 4-bit?
Uhereye kuri 16 bits kuburemere ukagera kuri 4 bits kugabanya ububiko hafi yikintu cya kane, niyo mpamvu moderi nini zishobora guhura na GPU imwe.
Ni ubuhe butumwa nyamukuru bwo kwibabaza buke-bito?
Gushushanya ibintu byinshi byagaciro kurwego ruto rwatakaje ibisobanuro, bishobora kugabanya ubuziranenge keretse gupima ubwenge birinda uburemere bworoshye.
Nigute amahugurwa-azi kubara atandukanye numubare wanyuma wamahugurwa?
Kwimenyereza kwimenyekanisha byubaka ikosa ryo kuzenguruka mu mahugurwa, bityo umuyoboro uhuza kandi mubisanzwe ugumana ukuri kwinshi mubugari buke.
Nubuhe buryo bukoresha ibipimo 4-biti kugirango uhindure neza moderi nini zihendutse kuri GPU imwe?
Q.