I-SmoothQuant kanye ne-activation Quantization
I-SmoothQuant yindlela eyenza kube nokwenzeka ukuminyanisa amamodeli olimi amakhulu kuze kufike kuzinombolo ezingu-8-bit zazo zombili izisindo nokwenza kusebenze ngaphandle kokuqeqeshwa kabusha.
Uhlolojikelele
It matters because activations in big models contain extreme outliers that normally wreck low-precision math, and SmoothQuant tames them.
I-Deep Dive
Uma unciphisa imodeli isuka kokuntanta okungu-16-bit iye ku-8-bit integers, izisindo ziminyanisa kalula kodwa ukwenza kusebenze kuyinkinga: iziteshi ezithile zithwala amanani amakhulu ngokuphindwe ka-10 kuye kwayi-100 kunamanye, futhi ukuwaphoqelela kugridi yenombolo ephelele kulimaza ukunemba. I-SmoothQuant, yethulwe ngu-Xiao et al. ngo-2022, ibona ukuthi izisindo zibushelelezi futhi kulula ukuzilinganisela ngenkathi ukwenza kusebenze kune-spiky. Ngakho-ke ithutha ubunzima ngezibalo: ihlukanisa iziteshi zokuvula ngesilinganiso seshaneli ngayinye futhi iphindaphinde izisindo ezihambisanayo ngesilinganiso esifanayo. Le misebenzi emibili iyakhansela, ishiya okukhiphayo kwemodeli kungashintshile, kodwa manje womabili ama-tensor asezindaweni ezinobungani. Umphumela uba i-W8A8 (izisindo eziyi-8-bit nokwenza kusebenze) ukulahleka kokunemba okucishe kube yiziro kanye nokusheshisa okulinganiselwa ku-2x nokonga inkumbulo.
I-Technical Insight
Iqhinga eliyinhloko into yokushelela kwesiteshi ngasinye s ehlanganiswe njengokuthi s = max(|X|)^alpha / max(|W|)^(1-alpha). Ukwenza kusebenze kukalwa ngo-1/s nezisindo ngo-s, ngakho umkhiqizo we-matrix XW uyalondolozwa. Ngenxa yokuthi isikali sithathwa ungaxhunyiwe ku-inthanethi ezisindweni zesendlalelo sangaphambilini noma umsebenzi ohlanganisiwe, sengeza izindleko zesikhathi sokusebenza eziyiziro. I-alpha hyperparameter (ngokuvamile engu-0.5) ilawula ukuthi ungakanani umthwalo ophuma ngaphandle osuka ekwenziweni kusebenze uye ezisindweni.
I-Strategic Impact
Izindleko kanye nesabelomali
Izinqumo zezakhiwo ziqhuba ukusebenza kanye nezindleko zokusebenza iminyaka.
Izinqumo ezicacile
Imfundo yobuchwepheshe isiza amaqembu ukuthi akhethe isitaki esifanele, hhayi nje esisha.
Ukulawulwa kwekhwalithi
Izinketho ezingcono zobunjiniyela zinciphisa izehlakalo ezinokwethenjelwa ekukhiqizeni.
Ikusasa Le-SmoothQuant kanye Nokwenza Kusebenze Ukulinganisa
I-SmoothQuant ithole ukuthi ama-outliers okwenza kusebenze ayafuduka kunokuba agwemeke, futhi lowo mbono manje usekela ukukhiqizwa kwe-INT8 ne-FP8. Lindela ukushelela ukuze kuhlanganiswe nezikimu ezinohlamvu olucolisekileyo njengokwenza inani leqembu ngalinye, ukukala okufundiwe, kanye nocwaningo lokuqalisa lwe-4-bit (isb. izindlela eziqaphelayo ngaphandle). Njengoba i-FP8 hardware (Hopper, Blackwell) ikhula, ukulinganisa kwesitayela okushelelayo kuzoqhubeka kubhakwa kumapayipi okuhlanganisa kanye nenjini yokucabanga ukuze ukulinganisa kuhlale kukhululekile.
Ukuqaliswa Komhlaba Wangempela
Ukukhonza i-LLM yepharamitha engu-70B ku-W8A8 kuma-GPU ambalwa ngokunciphisa kokubili inkumbulo nezindleko zokuphindaphinda kwe-matrix
Inika amandla i-INT8 inference kuma-NVIDIA Hopper/Blackwell tensor cores asheshisa ngokomdabu izibalo eziyi-8-bit integer
Isebenzisa amamodeli engxoxo ezindaweni zokugcina zamafu ezikhawulelwe izindleko lapho ukuphuma okuphindwe kabili kunciphisa ngokuqondile inkokhelo yethokheni ngayinye
Ukucindezela izifaki khodi ze-transformer enkulumweni ekudivayisi noma ukuhumusha lapho ama-8-bit kernels agijima ngokushesha nangokupholile.
Izingozi & Guardrails
Ukuthuthukisa ibhentshimakhi eyodwa kungafihla ubuthakathaka obubanzi besistimu.
Izindleko zengqalasizinda nezokulungisa zivame ukubukelwa phansi.
Izikhala zokuphepha nokubonakala zingakhula njengoba izinhlelo ziba nzima kakhulu.
Ukuqalisa Umhlahlandlela
Chaza ukubambezeleka, ikhwalithi, nezindleko ezihlosiwe ngaphambi kokuqaliswa.
Ibhentshimakhi ngaphansi komthwalo wangempela nezimo zedatha.
Ukuqapha amathuluzi amaphutha, ukukhukhuleka, nomthelela wabasebenzisi.
Lungiselela izindlela zokuhlehlisa nezigameko ngaphambi kokukala.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the SmoothQuant and Activation Quantization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Umhlahlandlela olandelayo
I-Activation Steering kanye Nobunjiniyela Bokumela
Imibuzo evame ukubuzwa
What is SmoothQuant and Activation Quantization?
I-SmoothQuant yindlela eyenza kube nokwenzeka ukuminyanisa amamodeli olimi amakhulu kuze kufike kuzinombolo ezingu-8-bit zazo zombili izisindo nokwenza kusebenze ngaphandle kokuqeqeshwa kabusha. Kubalulekile ngoba ukwenza kusebenze kumamodeli amakhulu kuqukethe izinto eziphuma ngaphandle ngokwedlulele ezivame ukulimaza izibalo ezinembayo, futhi i-SmoothQuant iyazithambisa.
Iyiphi inkinga i-SmoothQuant ebhekana nayo ngokuqondile ekuqondeni okuphansi?
I-SmoothQuant iqondise amanani angaphandle amakhulu avela eziteshini ezithile zokuvula, ezilimaza ukunemba uma ukwenza kusebenze kubalwa kumabhithi angu-8.
I-SmoothQuant ikwenza kanjani ukuthi kusebenze kube lula ukulinganisa?
Ihlukanisa iziteshi zokuvula ngesilinganiso sesiteshi ngasinye futhi iphindaphinde izisindo ezifanayo ngaleso sikali, ngakho umkhiqizo awushintshile kodwa womabili ama-tensor kuba lula ukuwalinganisa.
Isho ukuthini i-notation W8A8?
I-W8A8 isho ukulinganisa okungu-8-bit kuzo zombili izisindo (W) nokwenza kusebenze (A), uhlelo oluthi SmoothQuant luvumela ukulahleka kokunemba okuncane.
Kungani ukukala kwe-SmoothQuant kungangezeleli isikhathi sokusebenza?
Izikali ezishelelayo zigoqwa zibe izisindo zesendlalelo esandulelayo noma i-op ehlanganisiwe ngaphambi kwesikhathi, ngakho-ke akukho zibalo ezengeziwe ezenzekayo ngokucatshangelwa.
Iyiphi indima edlalwa yi-alpha hyperparameter ku-SmoothQuant?
I-Alpha (ngokuvamile engu-0.5) ibhalansisa into eshelelayo, inquma ukuthi bungakanani ubunzima bokulinganisa obususwa ekusebenzeni futhi kufakwe ezisindweni.