Nlereanya Quantization
Nhazi nke ụdịdị na-ebelata netwọkụ akwara ozi site na ịchekwa ọnụọgụ ya na ntakịrị ntakịrị, yabụ otu ụdị ahụ na-agba ọsọ ọsọ yana na ngwaike pere mpe.
Nchịkọta
It is the main reason large models can fit on a single GPU, a laptop, or even a phone.
Ime miri emi
Ụdị a zụrụ azụ na-echekwa ibu ọ bụla dị ka ọnụọgụ 32-bit ma ọ bụ 16-bit sere n'elu. Quantization na-anọchi ndị nwere ụdị nkenke dị ala dị ka 8-bit integers (INT8) ma ọ bụ ụkpụrụ 4-bit (INT4), na-egbutu ebe nchekwa ihe dịka 4x ruo 8x. Ihe atụ 70-ijeri-ijeri nke chọrọ ihe dịka 140GB na 16-bit nwere ike ịdaba nso 35GB na 4-bit, dabara na otu GPU ndị ahịa. Njide ahụ bụ izi ezi: ịpịnye ụkpụrụ dị iche iche n'ime bọket 256 ma ọ bụ 16 tụfuru nkọwa. Ụzọ ọgbara ọhụrụ dị ka GPTQ, AWQ, na usoro NF4 eji na QLoRA na-ahọrọ ihe ndị na-akpachapụ anya ma na-echebe ihe dị arọ kacha nwee mmetụta, ya mere ọnwụ dị mma na-adịkarị obere. Quantization bụ ya mere ngwaọrụ dị ka llama.cpp na Ollama nwere ike na-agba ọsọ ụdị nwere ike na mpaghara na-enweghị ebe data.
Nghọta nka nka
Ngụkọta maapụ ezi ụkpụrụ gaa na grid integer dị nta na-eji ọ̀tụ̀tụ̀ na ebe efu: echekwara_int = gburugburu(uru / ọnụ ọgụgụ) + zero_point. Ịhọrọ ọnụ ọgụgụ nke ọma bụ egwuregwu dum. Ntụle ọwa ọ bụla ma ọ bụ otu otu na-edobe akpịrịkpa dị iche iche maka mpekere matriks dị arọ, na-echekwa nkenke ebe ọ dị mkpa. Quantization nke ọzụzụ ọzụzụ gachara na-agbanwe ụdị emechachara, ebe ọzụzụ ịma mma nke quantization na-eme okirikiri n'oge ọzụzụ ka netwọkụ wee mụta ịnabata ya, na-enyekarị izi ezi nke obere obere.
Mmetụta atụmatụ
Ọnụ ego na mmefu ego
Mkpebi ihe owuwu ụlọ na-akwalite arụmọrụ yana ọnụ ahịa ọrụ ruo ọtụtụ afọ.
Mkpebi doro anya
Nkà mmụta nka na-enyere ndị otu egwuregwu aka ịhọrọ nchịkọta ziri ezi, ọ bụghị naanị nke kachasị ọhụrụ.
Quality akara
Nhọrọ injinia ka mma na-ebelata ihe omume ntụkwasị obi na mmepụta.
Ọdịnihu nke Nlereanya Quantization
Na-atụ anya na izi ezi na-adị ala ga-abụ nke nkịtị. Nnyocha na-akwalite 4-bit, 2-bit, na ọbụna ọnụọgụ ọnụọgụ abụọ, yana atụmatụ agwakọtara nke ọma na-eme ka ọkwa dị nro dị elu. Ngwaike na-eso: GPUs na ibe ekwentị ugbu a gụnyere nkeji INT8, INT4 na FP8 math. Ụdị dị ka FP8 na MXFP4 bu n'obi ikpokọta oke nke floats na nha nke integers. Ejikọta ya na usoro dị ka QLoRA, quantization ga-anọgide na-eme ka ụdị ọnụ ala dị ọnụ ala iji na-agba ọsọ na nke ọma na-emegharị na ngwaọrụ kwa ụbọchị.
Mmejuputa n'ezie n'ụwa
Na-agba ọsọ ụdị 7B ma ọ bụ 13B Llama na laptọọpụ na llama.cpp ma ọ bụ Ollama na-eji faịlụ GGUF 4-bit.
QLoRA na-edozi nnukwu ihe nlereanya na otu GPU site na idobe ihe dị ala na 4-bit NF4.
Na-ebuga ụdị INT8 na ekwentị nwere oge arụ ọrụ na-arụ ọrụ ka ndị enyemaka na-arụ ọrụ na-anọghị n'ịntanetị na nzuzo.
Ijere ebe njedebe API dị ọnụ ala karịa ebe ọnụọgụ INT8/FP8 na-agbadata ntinye ugboro abụọ ma belata ọnụ ahịa ebe nchekwa.
Ihe ize ndụ & okporo ụzọ nche
Ịkwalite otu akara ngosi nwere ike zoo adịghị ike sistemụ sara mbara.
A na-eledakarị ihe akụrụngwa na ụgwọ ọrụ anya.
Ọdịiche nchekwa na nleba anya nwere ike itolite ka sistemu na-adịwanye mgbagwoju anya.
Map mmejuputa
Kọwaa latency, ịdịmma na ebumnuche ọnụ ahịa tupu mmejuputa ya.
Benchmark n'okpuru ibu dị adị na ọnọdụ data.
Nleba anya akụrụngwa maka mperi, ịkpafu na mmetụta onye ọrụ.
Kwadebe ụzọ nzaghachi azụghachi azụ na ihe omume tupu ịchachaa.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Model Quantization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Ndebanye aha ụdị
Ajụjụ a na-ajụkarị
What is Model Quantization?
Nhazi nke ụdịdị na-ebelata netwọkụ akwara ozi site na ịchekwa ọnụọgụ ya na ntakịrị ntakịrị, yabụ otu ụdị ahụ na-agba ọsọ ọsọ yana na ngwaike pere mpe. Ọ bụ isi ihe kpatara nnukwu ụdị nwere ike dabara na otu GPU, laptọọpụ, ma ọ bụ ọbụna ekwentị.
Kedu ihe nleba anya ụdịdị na-agbanwe n'ụzọ bụ isi gbasara netwọkụ akwara?
Quantization na-echekwa otu paramita ahụ na ntakịrị ntakịrị (dịka ọmụmaatụ INT8 ma ọ bụ INT4 kama 16- ma ọ bụ 32-bit floats), na-ebelata ebe nchekwa na ịgba ọsọ mgbakọ na mwepụ.
Ebe nchekwa ole nwere ike ịtụgharị ihe nlereanya site na 16-bit gaa na 4-bit zọpụta?
Ịga site na 16 ibe n'ibe kwa arọ gaa na 4 bits na-ebelata nchekwa ihe dị ka ihe dị ka ihe anọ, nke mere na nnukwu ụdị nwere ike dabara na otu GPU.
Kedu ihe bụ isi ọdịda nke ọnụ ọgụgụ obere-bit ike ike?
Ịdepụta ụkpụrụ dị iche iche na ọkwa ole na ole pụrụ iche na-efunahụ nkọwa zuru ezu, nke nwere ike ibelata ogo ọ gwụla ma ịchacha mma na-echebe ịdị arọ dị nro.
Kedu ka ọzụzụ-ịmaara nke ọma si dị iche na ngụpụta nke ọzụzụ gachara?
Ọzụzụ mara nke ọma na-ewulite njehie okirikiri n'ime loop ọzụzụ, yabụ netwọk ahụ na-agbanwe ma na-edobekarị izi ezi na obere obosara.
Kedu usoro na-eji quantization 4-bit na-eme ka ọmarịcha ụdị dị mma dị ọnụ ala na otu GPU?
QLoRA na-edobe ihe nrịba ama n'ụdị 4-bit NF4 ma na-azụ obere ihe nkwụnye ọkụ, na-ahapụ nnukwu ụdị ka edobere ya nke ọma na ngwaike dị obere.