Quantization SmoothQuant iyo Hawlgelinta Tirinta
SmoothQuant waa farsamo suurtagelinaya in la isku cadaadiyo moodooyinka luqadaha waaweyn ilaa 8-bit integers labadaba miisaanka iyo hawlgelinta iyada oo aan dib loo tababarin.
Dulmar
It matters because activations in big models contain extreme outliers that normally wreck low-precision math, and SmoothQuant tames them.
quusid qoto dheer
Markaad ka yareyso moodal ka soo jeeda 16-bit oo sabeynaya ilaa 8-bit isteejka, miisaannada ayaa si fudud u cadaadiya laakiin hawlqabadyadu waa dhib: kanaalada qaarkood waxay wataan qiyamka 10 ilaa 100 jeer ka weyn inta soo hartay, oo ku qasbida shabaqyada isugeynta aan fiicneyn waxay burburisaa saxnaanta. SmoothQuant, waxaa soo bandhigay Xiao et al. sanadka 2022, waxa ay eegtaa in miisaanku uu siman yahay oo ay fududahay in la qiyaaso iyada oo hawlqabadyadu ay yihiin kuwo aad u sarreeya. Markaa xisaab ahaan waxay u guuraysaa dhibka: waxay u qaybisaa kanaalada firfircoonida cabirka kanaalka waxayna ku dhufataa miisaannada u dhigma isla miisaan. Labadii hawlgalba way baajiyeen, iyagoo ka tegaya wax-soo-saarka moodalka oo aan isbeddelin, laakiin hadda labada kiraysteba waxay ku fadhiyaan meelo saaxiibtinimo leh. Natiijadu waa W8A8 (miisaan 8-bit ah iyo hawlqabadyo) oo leh luminta saxda ah ee eber u dhow iyo qiyaasta xawaaraha 2x iyo kaydinta xusuusta.
Aragtida Farsamada
Xeeladda udub-dhexaadka ah waa qodobka fududaynta kanaalka kasta oo loo xisaabiyo sida s = max (|X|) ^ alpha / max (| W|) ^ (1-alpha). Dhaqdhaqaaqyada waxaa lagu qiyaasaa 1/s iyo culeysyada s, markaa badeecada shaxanka ee XW waa la xafiday. Sababtoo ah is-qarinta ayaa si toos ah loogu dhuuqay miisaanka lakabkii hore ama hawlgal isku dhafan, waxay ku daraysaa eber qiimaha runtime. Halbeegga alfa hyperparameter (badanaa 0.5) ayaa xakameysa inta culeyska ka baxsan uu ka soo wareego dhaqdhaqaaqyada culeysyada.
Saamaynta Istiraatijiyadeed
Qiimaha iyo miisaaniyada
Go'aamada qaab-dhismeedku waxay horseedaan waxqabadka iyo kharashka hawlgalka sannadaha.
Go'aamo cad
Waxbarashada farsamada waxay ka caawisaa kooxaha inay doortaan xidhmo sax ah, ma aha oo kaliya kan ugu cusub.
Xakamaynta tayada
Doorashooyinka injineernimada ee wanaagsan waxay yareeyaan shilalka la isku halleyn karo ee wax soo saarka.
Mustaqbalka Quant-ka SmoothQuant iyo Tirinta Dhaqdhaqaaqa
SmoothQuant waxa ay dejisay in hawl-gelintu tahay guuri halkii laga fursan lahaa, fikraddaasina hadda waxa ay taageertaa wax soo saarka INT8 iyo FP8. Filo fududaynta in lagu daro qorshayaal miro-dhal ah sida koox-koox kasta, xisaabin la bartay, iyo cilmi-baadhis hawleed 4-bit ah (tusaale, hababka ka-ogsoon). Sida qalabka FP8 (Hopper, Blackwell) u bislaado, isu dheelitirka-qaabka siman waxa lagu sii duban doonaa tuubooyinka mishiinada isku duba ridka si ay tiradu u ahaato mid bilaash ah.
Dhaqangelinta Adduunka-dhabta ah
U adeegida 70B-parameter LLM ee W8A8 GPU-yo yar iyada oo la kala badhayo labadaba xusuusta iyo matrix-ku-dhufashada qiimaha
Awood u siinta fikradda INT8 ee NVIDIA Hopper/Blackwell tensor cores kuwaas oo si asal ah u dardargeliya xisaabta 8-bit
Gelitaanka moodooyinka wada sheekaysiga ee meelaha daruuriga ah ee kharashkoodu xaddidan yahay halkaas oo laba-laabashada wax-soo-saarka ay si toos ah u jarayso biilka-calaamadaha
Isku-buufinta qalabka wax-ka-beddelka ee hadalka qalabka ama tarjumaadda halkaasoo 8-bit kernels ay u socdaan si degdeg ah oo qabow.
Khatarta & Dariiqyada Ilaalada
Hagaajinta hal bartilmaameed waxay qarin kartaa daciifnimada nidaamka ballaaran.
Kaabayaasha dhaqaalaha iyo dayactirka inta badan waa la dhayalsadaa.
Nabadgelyada iyo daldaloolada u fiirsashada ayaa kori kara marka nidaamyadu noqdaan kuwo aad u adag.
Qorshe Hawleedka Dhaqangelinta
Qeex daahida, tayada, iyo bartilmaameedyada qiimaha ka hor inta aan la hirgelin.
Benchmark marka la eego culeyska dhabta ah iyo xaaladaha xogta.
La socodka qalabka khaladaadka, leexashada, iyo saamaynta isticmaalaha.
U diyaari dib-u-noqoshada iyo dariiqyada jawaab-celinta dhacdada ka hor inta aanad miisaan.
Sii wad Sahaminta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the SmoothQuant and Activation Quantization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Hagaha xiga
Hagida Hawlgelinta iyo Injineernimada Matalaada
Su'aalaha soo noqnoqda
What is SmoothQuant and Activation Quantization?
SmoothQuant waa farsamo suurtagelinaya in la isku cadaadiyo moodooyinka luqadaha waaweyn ilaa 8-bit integers labadaba miisaanka iyo hawlgelinta iyada oo aan dib loo tababarin. Waa arrin sababtoo ah hawlgelinta moodooyinka waaweyn waxay ka kooban yihiin waxyaabo aad uga baxsan kuwaas oo sida caadiga ah burburiya xisaabta saxda ah ee hooseeya, oo SmoothQuant iyaga ayaa baraya.
Dhibaato noocee ah ayuu SmoothQuant si gaar ah uga hadlayaa fikradda saxda ah ee hoose?
SmoothQuant waxay beegsataa qiyamka bannaanka weyn ee ka muuqda kanaalada hawlgelinta qaarkood, taas oo haddii kale baabi'isa saxnaanta marka hawlgelinta lagu qiyaaso 8 bits.
Sidee ayay SmoothQuant u fududaynaysaa in la qiyaaso?
Waxa ay u qaybisaa kanaalada firfircoonida iyada oo loo eegayo halbeeg-kanaal ah waxana ay ku dhufataa miisaannada isbarbardhigga ah ee miisaankaas, markaa alaabtu isma beddelin laakiin labada kiraysteba way fududahay in la qiyaaso.
Waa maxay macnaha qoraalka W8A8?
W8A8 waxay muujinaysaa qiyaasida 8-bit ee labada miisaan (W) iyo firfircoonida (A), nidaamka SmoothQuant wuxuu awood u leeyahay luminta saxnaanta ugu yar.
Waa maxay sababta miisaanka SmoothQuant ay ugu darayso runtime wax ka badan?
Miisaanka fududaynta waxa lagu laalaa miisanka lakabka hore ama ikhtiyaar la isku daray ka hor, markaa xisaabin dheeraad ah ma dhacdo marka la eego.
Doorkee ayuu ka ciyaaraa alfa hyperparameterka gudaha SmoothQuant?
Alpha (caadi ahaan 0.5) waxay dheellitiraysaa qodobka fududaynta, go'aaminaaya inta ay le'eg tahay dhibka xisaabinta ayaa laga raray dhaqdhaqaaqyada iyo miisaanka.