Quantization SmoothQuant na ịgbalite
SmoothQuant bụ usoro na-eme ka o kwe omume ịpịkọta nnukwu ụdị asụsụ ruo 8-bit integers maka ma ọ̀tụ̀tụ̀ ịdị arọ na n'arụ ọrụ na-abụghị ọzụzụ.
Nchịkọta
It matters because activations in big models contain extreme outliers that normally wreck low-precision math, and SmoothQuant tames them.
Ime miri emi
Mgbe ị na-ebelata ihe nlereanya site na 16-bit na-ese n'elu ruo 8-bit integers, ịdị arọ na-agbakọ ngwa ngwa ma ọrụ ọrụ bụ nsogbu: ụfọdụ ọwa na-ebu ụkpụrụ 10 ruo 100 ugboro karịa ndị ọzọ, na ịmanye ha n'ime grid integer na-adịghị mma na-ebibi izi ezi. SmoothQuant, nke Xiao et al webatara. na 2022, na-achọpụta na arọ dị ire ụtọ na mfe quantize mgbe activation bụ spiky. Ya mere ọ na mgbakọ na mwepụ na-akwaga ihe isi ike: ọ na-ekewa ọwa ịgbalite site na ọnụọgụ ọwa ọ bụla ma na-amụba nha kwekọrọ n'otu nha. Ọrụ abụọ ahụ kagburu, na-ahapụ mmepụta ihe nlereanya agbanweghị, ma ugbu a ma tenors na-anọdụ ala na enyi enyi. Nsonaazụ bụ W8A8 (8-bit weights and activations) inference na nso-efu ziri ezi ọnwụ na ihe enyemaka 2x ọsọ ọsọ na nchekwa nchekwa.
Nghọta nka nka
Isi aghụghọ bụ ihe na-eme ka ọ bụrụ ọwa na-agbakọ dị ka s = max (| X|) ^ alpha / max (| W |) ^ (1-alpha). A na-eji 1/s tụọ ihe nrụ ọrụ yana ihe ọ̀tụ̀tụ̀ site na s, ya mere a na-echekwa ngwaahịa matrix XW. N'ihi na a na-etinye ihe nchacha ahụ na-anọghị n'ịntanetị n'ime ihe dị arọ nke oyi akwa gara aga ma ọ bụ ọrụ ejikọtara ọnụ, ọ na-agbakwunye ọnụ ahịa oge efu efu. Alfa hyperparameter (mgbe 0.5) na-achịkwa ole ibu dịpụrụ adịpụ na-agbanwe site na ịgbalite gaa n'ịdị arọ.
Mmetụta atụmatụ
Ọnụ ego na mmefu ego
Mkpebi ihe owuwu ụlọ na-akwalite arụmọrụ yana ọnụ ahịa ọrụ ruo ọtụtụ afọ.
Mkpebi doro anya
Nkà mmụta nka na-enyere ndị otu egwuregwu aka ịhọrọ nchịkọta ziri ezi, ọ bụghị naanị nke kachasị ọhụrụ.
Quality akara
Nhọrọ injinia ka mma na-ebelata ihe omume ntụkwasị obi na mmepụta.
Ọdịnihu nke SmoothQuant na ntinye ọrụ
SmoothQuant kwuputara na ndị na-arụ ọrụ na-apụ apụ bụ ihe a na-apụghị izere ezere, na echiche ahụ na-akwado mmepụta INT8 na FP8. Na-atụ anya ka ejikọta ọnụ ire ụtọ na atụmatụ dị mma dị ka ọnụọgụ nke otu ọ bụla, nlegharị anya mmụta, na nyocha 4-bit (dịka ụzọ mara mma). Ka ngwaike FP8 (Hopper, Blackwell) na-etolite, nhazi ụdị edozi ga-aga n'ihu na-esi ya n'ime ọkpọkọ igwe na igwe inference ka ọnụọgụ ga-anọrịrị n'efu.
Mmejuputa n'ezie n'ụwa
Ije ozi 70B-parameter LLM na W8A8 na GPU ole na ole site na ibelata ma ebe nchekwa na matriks-na-amụba ọnụ ahịa
Na-eme ka ntinye aka INT8 na NVIDIA Hopper/Blackwell tensor cores nke na-eme ka mgbakọ na mwepụ 8-bit dị ngwa ngwa.
Na-ebunye ụdị nkata na njedebe igwe ojii na-amachibido ọnụ ahịa ebe ntinye ntinye okpukpu abụọ na-ebipụ ụgwọ ụgwọ otu-token ozugbo.
Na-akpakọ ihe ngbanwe mgbanwe maka okwu ma ọ bụ ntụgharị ngwaọrụ na ngwaọrụ ebe kernel 8-bit na-agba ọsọ ọsọ na oyi.
Ihe ize ndụ & okporo ụzọ nche
Ịkwalite otu akara ngosi nwere ike zoo adịghị ike sistemụ sara mbara.
A na-eledakarị ihe akụrụngwa na ụgwọ ọrụ anya.
Ọdịiche nchekwa na nleba anya nwere ike itolite ka sistemu na-adịwanye mgbagwoju anya.
Map mmejuputa
Kọwaa latency, ịdịmma na ebumnuche ọnụ ahịa tupu mmejuputa ya.
Benchmark n'okpuru ibu dị adị na ọnọdụ data.
Nleba anya akụrụngwa maka mperi, ịkpafu na mmetụta onye ọrụ.
Kwadebe ụzọ nzaghachi azụghachi azụ na ihe omume tupu ịchachaa.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the SmoothQuant and Activation Quantization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Nhazi ọrụ na nnochite anya injinia
Ajụjụ a na-ajụkarị
What is SmoothQuant and Activation Quantization?
SmoothQuant bụ usoro na-eme ka o kwe omume ịpịkọta nnukwu ụdị asụsụ ruo 8-bit integers maka ma ọ̀tụ̀tụ̀ ịdị arọ na n'arụ ọrụ na-abụghị ọzụzụ. Ọ dị mkpa n'ihi na ịgbalite na nnukwu ụdị nwere ihe dị oke egwu nke na-emebikarị mgbakọ na mwepụ dị ala, yana SmoothQuant na-azụ ha.
Kedu nsogbu SmoothQuant na-ekwu kpọmkwem na ntinye nkenke dị ala?
SmoothQuant na-elekwasị anya na nnukwu ụkpụrụ dịpụrụ adịpụ nke na-apụta na ọwa mmemme ụfọdụ, nke na-emebi izi ezi ma ọ bụrụ na agbanyere ọrụ ka ọ bụrụ 8 bits.
Kedu ka SmoothQuant si eme ka mmemme dị mfe ịgụta ọnụ?
Ọ na-ekewa ọwa ịgbalite site n'ọ̀tụ̀tụ̀ ọ̀tụ̀tụ̀ ọ bụla ma mee ka ọ̀tụ̀tụ̀tụ̀ dabara adaba na-amụba site n'ọnụọgụ ahụ, ya mere ngwaahịa ahụ agbanweghị agbanwe mana tenors abụọ ahụ na-adị mfe ịgụta ọnụ.
Kedu ihe akara W8A8 pụtara?
W8A8 na-egosi ọnụọgụ 8-bit maka nha abụọ (W) na ịgbalite (A), ọchịchị SmoothQuant na-enyere ya aka na mfu kacha mma.
Kedu ihe kpatara SmoothQuant's scaling na-agbakwụnye n'ezie enweghị oge ịgba ọsọ?
A na-apịaji ihe ọ̀tụ̀tụ̀ ire ụtọ n'ime ihe ọ̀tụ̀tụ̀ dị n'ígwé ndị bu ụzọ ma ọ bụ ihe ejikọtara ọnụ tupu oge eruo, n'ihi ya, ọ dịghị mgbakọ ọzọ na-eme n'echiche.
Kedu ọrụ alfa hyperparameter na-arụ na SmoothQuant?
Alfa (nke a na-ahụkarị 0.5) na-edozi ihe na-eme ka ọ dị nro, na-ekpebi ole n'ime ihe isi ike quantization na-ewepụ n'ọrụ na n'elu igwe.