Quantization
Quantization na-ebelata ihe nlereanya AI site na ịchekwa nọmba ya na nkenke dị ala, yabụ ụdị nke chọrọ GPU etiti data nwere ike na-agba ọsọ na laptọọpụ ma ọ bụ ekwentị mgbe ụfọdụ.
Nchịkọta
It is the main trick that makes large language models cheap and fast enough to deploy widely.
Ime miri emi
Netwọk akwara na-abụkarị nnukwu ọnụọgụ ọnụọgụ nke a na-akpọ ịdị arọ, nke a na-echekwakarị ka ụkpụrụ 16- ma ọ bụ 32-bit na-ese n'elu mmiri. Quantization na-echekwa nha ndị ahụ site na iji obere ibe n'ibe, ọtụtụ 8-bit (INT8) ma ọ bụ ọbụna ọnụọgụ 4-bit. Site na 16-bit gaa na 4-bit na-ebelata ebe nchekwa ihe dị ka okpukpu anọ, yabụ ụdị paramita ijeri 70 nke chọrọ ihe dịka 140GB na 16-bit nwere ike dabara na 35GB na 4-bit. Ọnụọgụ ndị nta na-agakwa na ebe nchekwa ngwa ngwa, nke na-emekarị ọgbọ na-agba ọsọ ọsọ. Njide ahụ bụ izi ezi: ịmanye ọtụtụ ụkpụrụ n'ime ọkwa ole na ole na-ewebata njehie okirikiri. Ụzọ dị mma na-ebelata mfu ahụ site n'iji nlezianya họrọ ihe ndị na-eme ka ọkpụkpụ na-echekwa ihe ndị dị nro dị nro, ya mere ihe nlereanya ahụ na-eme ihe fọrọ nke nta ka ọ bụrụ otu mgbe ọ na-eji ntakịrị ihe onwunwe.
Nghọta nka nka
Otu ọ bụla nke ihe ọ̀tụ̀tụ̀ ọ̀tụ̀tụ̀ ọ̀tụ̀tụ̀ ọ̀tụ̀tụ̀ ọ̀tụ̀tụ̀ ọ̀tụ̀tụ̀ ọ̀tụ̀tụ̀ ọ̀tụ̀tụ̀ ọ̀tụ̀tụ̀ ọ̀tụ̀tụ̀ ọ̀tụ̀tụ̀ ọ̀tụ̀tụ̀ ọ̀tụ̀tụ̀ ọ̀tụ̀tụ̀ ọ̀tụ̀tụ̀ ọ̀tụ̀tụ̀ bụ́ nke na-esetịpụ ụkpụrụ n'ezie n'obere ọnụ ọgụgụ dị nta; amụba azụ site n'ọ̀tụ̀tụ̀ dị ka ọ na-ewughachi nọmba mbụ ahụ. Ụzọ ngụpụta ngụsị akwụkwọ mgbe ọzụzụ dị ka GPTQ na AWQ na-enyocha obere dataset mmezi iji kpebie nke kachasị mkpa ma tọọ akpịrịkpa iji belata njehie mmepụta, kama ịmegharị ihe niile na-enweghị isi. A na-edobe mmemme n'ọkwa dị elu n'ihi na ha na-adịgasị iche karịa n'oge ọsọ. Nsonaazụ bụ ụdị na-echekwa integers 4-bit mana na-agbakọ nsonaazụ dị nso na ụdị zuru oke.
Mmetụta atụmatụ
Ọsọ na ọnụ ọgụgụ
Usoro ọrụ asụsụ nwere ike ịga ngwa ngwa n'achụghị nkwụsi ike.
Nweta na iru
Ọ na-agbasawanye ohere n'ofe asụsụ na ụdị nzikọrịta ozi.
Mkpebi doro anya
Ndị otu nwere ike itinyekwu oge na ikpe ebe akpaaka na-ejikwa nkwughachi.
Ọdịnihu nke Quantization
Na-atụ anya quantization ga-abụ nke ndabara kama nkwalite. Ndị na-ere ngwaike na-agbakwunye nwa amaala 4-bit na ọbụna nkwado nke obere-bit, yana usoro dị ka anabatara ọzụzụ na-eme achịcha maka obere nkenke n'ime ihe nlereanya site na mmalite, na-ebelata nkwụsi ike n'ihu. Nnyocha n'ime ihe ngosi 2-bit na 1-bit (ọnụọgụ abụọ) na-arụsi ọrụ ike, na-achọ ịme ụdị nwere ike na ekwentị yana ibe agbakwunyere. Dị ka ngwaọrụ na AI nkeonwe na-eto eto, ụdị ọnụọgụgụ nke ọma ga-abụ isi maka ndị na-enyere aka na mpaghara na-ezigaghị data na igwe ojii.
Mmejuputa n'ezie n'ụwa
Na-agba ọsọ ụdị nkata dị ka Llama na mpaghara na GPU onye ahịa na-eji faịlụ 4-bit GGUF ma ọ bụ GPTQ kama ịchọ kaadị etiti data ọtụtụ.
Ndị na-enyere aka na ngwaọrụ na ekwentị, ebe ụdị 8-bit ma ọ bụ 4-bit na-ahapụ atụmatụ okwu na ederede na-arụ ọrụ na-enweghị njikọ netwọkụ.
Ịbelata ntinye aka igwe ojii maka bot nkwado ndị ahịa site n'ijere ụdị INT8, na-adaba arịrịọ ndị ọzọ na GPU ọ bụla.
Ngwa Edge dị ka igwefoto smart ma ọ bụ ihe mmetụta IoT na-agba ụdị asụsụ ọhụụ kọmpat n'ime oke ebe nchekwa.
Ihe ize ndụ & okporo ụzọ nche
Eziokwu ndị e chepụtara echepụta nwere ike jiri nwayọ tinye akụkọ, nkwado nkwado, ma ọ bụ nsonaazụ nyocha.
Mmetụta ngwa ngwa nwere ike ịmepụta nsonaazụ na-ekwekọghị ekwekọ n'ofe arịrịọ ndị yiri ya.
Enwere ike ikpughe data ederede nwere mmetụta ma ọ bụrụ na njikwa ohere adịghị ike.
Map mmejuputa
Kọwaa usoro mmepụta, ụda, na ụkpụrụ ịdịmma tupu ibugharị.
Weghachite nzaghachi site na isi mmalite ntụkwasị obi mgbe ọ bụla izi ezi dị mkpa.
Debe ebe nleba anya mmadụ maka mpụta dị elu.
Sochie ụkpụrụ ọdịda ma na-azụghachi mkpali ma ọ bụ usoro ọrụ mgbe niile.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Quantization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Ngụkọta akụkụ nke vector fọdụrụnụ
Ajụjụ a na-ajụkarị
What is Quantization?
Quantization na-ebelata ihe nlereanya AI site na ịchekwa nọmba ya na nkenke dị ala, yabụ ụdị nke chọrọ GPU etiti data nwere ike na-agba ọsọ na laptọọpụ ma ọ bụ ekwentị mgbe ụfọdụ. Ọ bụ aghụghọ bụ isi na-eme ka ụdị asụsụ buru ibu dị ọnụ ala yana ngwa ngwa zuru oke iji tinye ebe niile.
Kedu ihe ọnụọgụ ọnụọgụ na-agbanwe n'ụzọ bụ isi gbasara netwọkụ akwara?
Quantization na-echekwa nha ihe nlereanya na nzizi ọnụọgụ dị ala, dị ka integers 8-bit ma ọ bụ 4-bit kama 16- ma ọ bụ 32-bit sere n'elu.
Kedu ihe nchekwa ole echekwara site na ibugharị arọ site na 16-bit gaa na 4-bit?
4 ibe n'ibe bụ otu ụzọ n'ụzọ anọ nke 16 bits, ya mere nchekwa arọ na-adaba na ihe dị ka otu ụzọ n'ụzọ anọ, ihe dị ka mbelata 4x.
Kedu ihe bụ isi ọdịda nke ọnụ ọgụgụ ike ike?
Na-anọchi anya ụkpụrụ nwere ọkwa dị nta na-ewebata njehie okirikiri, nke nwere ike weda ogo mmepụta ma ọ bụrụ na ejirighị ya nke ọma.
Kedu ụzọ dị ka GPTQ na AWQ na-eji obere nhazi dataset maka?
Ụzọ ọzụzụ ndị a gachara na-enyocha ntinye nlele ole na ole iji tọọ ihe na-akpachapụ anya ma chebe ịdị arọ ndị nwere mmetụta, na-edobe nsonaazụ nso nke mbụ.
Gịnị mere quantization na-emekarị ihe nlereanya ngwa ngwa, ọ bụghị naanị obere?
Ụkpụrụ ziri ezi dị ala na-ewe obere bandwidth ebe nchekwa iji buru ibu ma kwaga, nke na-abụkarị ihe mgbochi n'oge ọgbọ, ya mere ntinye aka na-agba ọsọ.