SmoothQuant da Ƙididdiga Kunnawa
SmoothQuant wata dabara ce da ke ba da damar damfara manyan nau'ikan harshe har zuwa intigers 8-bit don duka nauyi da kunnawa ba tare da sake horarwa ba.
Dubawa
It matters because activations in big models contain extreme outliers that normally wreck low-precision math, and SmoothQuant tames them.
Zurfafa nutsewa
Lokacin da ka rage samfurin daga 16-bit yana yawo zuwa lamba 8-bit, ma'aunin nauyi yana daɗaɗa sauƙi amma kunnawa yana da matsala: wasu tashoshi suna ɗaukar ƙimar sau 10 zuwa 100 fiye da sauran, kuma tilasta su cikin grid mai ƙima yana lalata daidaito. SmoothQuant, wanda Xiao et al ya gabatar. a cikin 2022, ya lura cewa ma'aunin nauyi yana da santsi kuma mai sauƙin ƙididdigewa yayin kunnawa yana da kauri. Don haka a cikin lissafi yana ƙaura wahalar: yana rarraba tashoshin kunnawa ta hanyar ma'aunin kowane tashoshi kuma yana ninka ma'aunin ma'auni daidai da sikelin iri ɗaya. Ayyukan biyu sun soke, suna barin fitowar samfurin baya canzawa, amma yanzu duka tenors suna zaune a cikin jeri na abokantaka. Sakamakon shine W8A8 (ma'auni 8-bit da kunnawa) tare da asarar daidaito kusa-sifiri da kusan saurin 2x da ajiyar ƙwaƙwalwar ajiya.
Fahimtar Fasaha
Babban abin da za'a yi shi ne ma'aunin santsi na kowane tashoshi wanda aka lissafta shi azaman s = max(|X|)^alpha / max(|W|)^(1-alpha). Ana yin ma'auni na kunnawa da 1/s kuma ma'auni ta s, don haka ana adana samfurin matrix XW. Saboda ana shigar da sikelin a layi a cikin ma'aunin ma'aunin Layer na baya ko aiki mai haɗaɗɗiya, yana ƙara farashin lokacin aiki sifili. Alfa hyperparameter (sau da yawa 0.5) yana sarrafa nawa nauyin nauyi ya tashi daga kunnawa zuwa nauyi.
Dabarun Tasiri
Kudin da kasafin kuɗi
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Shawarwari masu haske
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Kula da inganci
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Makomar SmoothQuant da Ƙididdiga Kunnawa
SmoothQuant ya tabbatar da cewa masu fitar da kunnawa suna iya ƙaura maimakon abin da ba za a iya kaucewa ba, kuma wannan ra'ayin yanzu yana ƙarfafa samar da INT8 da FP8 hidima. Yi tsammanin za a haɗa santsi tare da ingantattun tsare-tsare kamar ƙididdige kowane rukuni, ƙididdige ƙididdigewa, da bincike na kunnawa 4-bit (misali hanyoyin da ba a sani ba). Yayin da kayan aikin FP8 (Hopper, Blackwell) ke balaga, daidaita-style mai santsi za a ci gaba da toya shi cikin bututun mai tarawa da injin ingin don haka ƙididdigewa ya kasance kusan kyauta.
Aiwatar da Gaskiyar Duniya
Hidimar 70B-parameter LLM a W8A8 akan ƴan GPUs ta hanyar rage duka ƙwaƙwalwar ajiya da matrix-yawan farashin
Ƙaddamar da bayanin INT8 akan NVIDIA Hopper/Blackwell tensor cores wanda ke haɓaka math 8-bit na asali.
Aiwatar da samfuran taɗi akan wuraren ƙarshe na girgije mai ƙayyadaddun farashi inda yawan abubuwan da aka samu sau biyu ke yanke lissafin kowane-alama kai tsaye.
Matsa maɓallan masu canza canji don magana a kan na'urar ko fassarar inda kernels 8-bit ke gudana cikin sauri da sanyi.
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the SmoothQuant and Activation Quantization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Gudanar da Kunnawa da Injiniya Wakili
Tambayoyin da ake yawan yi
What is SmoothQuant and Activation Quantization?
SmoothQuant wata dabara ce da ke ba da damar damfara manyan nau'ikan harshe har zuwa intigers 8-bit don duka nauyi da kunnawa ba tare da sake horarwa ba. Yana da mahimmanci saboda kunnawa a cikin manyan samfura suna ƙunshe da matsananciyar ƙetare waɗanda galibi suna lalata math ɗin ƙima, kuma SmoothQuant yana horar da su.
Wace matsala SmoothQuant ke magance ta musamman a cikin ƙayyadaddun ƙima?
SmoothQuant yana harba manyan ma'auni masu ma'ana waɗanda ke bayyana a wasu tashoshi na kunnawa, waɗanda in ba haka ba suna lalata daidaito lokacin da aka ƙididdige kunnawa zuwa 8 bits.
Ta yaya SmoothQuant ke sauƙaƙe kunnawa don ƙididdigewa?
Yana raba tashoshi kunnawa ta hanyar kowane tashoshi kuma yana ninka ma'aunin ma'auni ta wannan sikelin, don haka samfurin baya canzawa amma duka tenors sun zama masu sauƙin ƙididdigewa.
Menene ma'anar alamar W8A8?
W8A8 yana nuna ƙididdige 8-bit don duka ma'aunin nauyi (W) da kunnawa (A), tsarin mulki SmoothQuant yana ba da damar asara kaɗan kaɗan.
Me yasa SmoothQuant's scaling yana ƙara da gaske ba tare da wuce gona da iri ba?
Ana naɗe ma'aunin santsi a cikin ma'aunin ma'aunin da ya gabata ko kuma abin da aka haɗa kafin lokaci, don haka babu ƙarin ƙididdigewa da zai faru idan aka kwatanta.
Wace rawa alpha hyperparameter ke takawa a cikin SmoothQuant?
Alpha (yawanci 0.5) yana daidaita ma'auni mai santsi, yana yanke shawarar yadda yawancin wahalar ƙididdigewa ke motsa daga kunnawa da kan ma'aunin nauyi.