Kwiyoroshya no Kuringaniza Ibikorwa
SmoothQuant nubuhanga butuma bishoboka guhuza imiterere yururimi runini kugeza kuri 8-bit integer kuburemere ndetse no gukora utabanje kwitoza.
Incamake
It matters because activations in big models contain extreme outliers that normally wreck low-precision math, and SmoothQuant tames them.
Kwibira cyane
Iyo ugabanije icyitegererezo kuva 16-bit kureremba kugeza kuri 8-bit integer, uburemere buragabanuka byoroshye ariko ibikorwa nibibazo: imiyoboro imwe itwara indangagaciro zikubye inshuro 10 kugeza 100 kurenza izindi, hanyuma ukayihatira muri gride yuzuye integer yangiza ukuri. SmoothQuant, yatangijwe na Xiao n'abandi. muri 2022, abona ko uburemere bworoshye kandi bworoshye kubara mugihe ibikorwa ari spiky. Imibare rero yimura ingorane: igabanya imiyoboro yo gukora kumurongo wa buri muyoboro kandi igwiza uburemere bujyanye nubunini bumwe. Ibikorwa byombi birahagarika, hasigara icyitegererezo gisohoka kidahindutse, ariko ubu tensors zombi zicaye murwego rwinshuti. Igisubizo ni W8A8 (uburemere bwa 8-bit na activation) hamwe no gutakaza hafi ya zeru no gutakaza hafi 2x kwihuta no kuzigama kwibuka.
Ubushishozi
Amayeri yibanze ni umuyoboro uhuza ibintu s ubarwa nka s = max (| X |) ^ alpha / max (| W |) ^ (1-alpha). Ibikorwa bipimwa na 1 / s hamwe nuburemere na s, bityo ibicuruzwa bya matrix XW birabitswe. Kuberako igipimo cyinjijwe kumurongo muburemere bwibice byabanjirije cyangwa ibikorwa byahujwe, byongera igiciro cya zeru. Alpha hyperparameter (akenshi 0.5) igenzura uburyo umutwaro wo hanze uva mubikorwa ugana ibiro.
Ingaruka z'Ingamba
Igiciro na bije
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo bisobanutse
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Kugenzura ubuziranenge
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Ejo hazaza ha SmoothQuant na Quantisation yibikorwa
SmoothQuant yashyizeho ko ibikorwa byo kwimuka byimuka aho kwirindwa, kandi icyo gitekerezo gishimangira umusaruro INT8 na FP8. Tegereza koroshya guhuzwa hamwe na gahunda nziza cyane nka buri tsinda rigereranya, wize gupima, hamwe nubushakashatsi bwa 4-bit (urugero: uburyo bwo kumenya neza). Mugihe ibyuma bya FP8 (Hopper, Blackwell) bimaze gukura, kuringaniza-uburyo bwo kuringaniza bizakomeza gutekwa mumashanyarazi hamwe na moteri ya moteri kugirango kubara bigume hafi yubusa.
Gushyira mu bikorwa Isi
Gukora 70B-parameter LLM kuri W8A8 kuri GPU nkeya mugabanya kabiri kwibuka na matrix-kugwiza igiciro
Gushoboza INT8 umwanzuro kuri NVIDIA Hopper / Blackwell tensor cores kavukire yihutisha imibare 8-biti imibare
Kohereza moderi yo kuganira kumurongo wigiciro cyigabanije aho ibicuruzwa byikubye kabiri bigabanya byimazeyo fagitire
Gucomeka kodegisi ya transformateur kumvugo cyangwa kubisobanuro aho intoki 8-biti yihuta kandi ikonje
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the SmoothQuant and Activation Quantization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Gukora Ibikorwa no Guhagararira Ubwubatsi
Ibibazo bikunze kubazwa
What is SmoothQuant and Activation Quantization?
SmoothQuant nubuhanga butuma bishoboka guhuza imiterere yururimi runini kugeza kuri 8-bit integer kuburemere ndetse no gukora utabanje kwitoza. Nibyingenzi kuko ibikorwa muri moderi nini zirimo gusohoka cyane mubisanzwe bisenya imibare mike-idasobanutse, kandi SmoothQuant irabahindura.
Ni ikihe kibazo SmoothQuant ikemura byumwihariko muburyo buke?
SmoothQuant yibasiye indangagaciro nini zo hanze zigaragara mumiyoboro imwe yo gukora, ubundi isenya ubunyangamugayo mugihe ibikorwa bigereranijwe kuri 8 bit.
Nigute SmoothQuant yorohereza ibikorwa byoroshye kubara?
Igabanya imiyoboro ya enterineti ku gipimo cya buri muyoboro kandi igwiza uburemere buhuye nicyo gipimo, bityo ibicuruzwa ntibihinduka ariko tensor zombi zoroha kubara.
Inyandiko W8A8 isobanura iki?
W8A8 yerekana umubare wa biti 8 kubipimo byombi (W) no gukora (A), ubutegetsi SmoothQuant butuma habaho gutakaza neza.
Kuki igipimo cya SmoothQuant cyongeraho mubyukuri nta gihe cyo gukora hejuru?
Umunzani woroshye wiziritse muburemere bwibice byabanjirije cyangwa op yahujwe mbere yigihe, kubwibyo rero nta kubara kwinyongera bibaho.
Ni uruhe ruhare alpha hyperparameter igira muri SmoothQuant?
Alpha (mubisanzwe 0.5) iringaniza ibintu byoroshya, ihitamo ingano yingorabahizi yo kwimurwa ikuwe mubikorwa no ku buremere.