Ubuyobozi bwa tekiniki

Kwiyoroshya no Kuringaniza Ibikorwa

SmoothQuant nubuhanga butuma bishoboka guhuza imiterere yururimi runini kugeza kuri 8-bit integer kuburemere ndetse no gukora utabanje kwitoza.

Umva iki gitaboAI-byavuzwe · ~2 min

Kuriyi page2 min soma
  1. Incamake
  2. Kwibira cyane
  3. Gushyira mu bikorwa Isi
  4. Ejo hazaza ha SmoothQuant na Quantisation yibikorwa
  5. Kubishyira mubikorwa
  6. Komeza Ubushakashatsi
  7. Ibibazo bikunze kubazwa

Incamake

It matters because activations in big models contain extreme outliers that normally wreck low-precision math, and SmoothQuant tames them.

Kwibira cyane

Iyo ugabanije icyitegererezo kuva 16-bit kureremba kugeza kuri 8-bit integer, uburemere buragabanuka byoroshye ariko ibikorwa nibibazo: imiyoboro imwe itwara indangagaciro zikubye inshuro 10 kugeza 100 kurenza izindi, hanyuma ukayihatira muri gride yuzuye integer yangiza ukuri. SmoothQuant, yatangijwe na Xiao n'abandi. muri 2022, abona ko uburemere bworoshye kandi bworoshye kubara mugihe ibikorwa ari spiky. Imibare rero yimura ingorane: igabanya imiyoboro yo gukora kumurongo wa buri muyoboro kandi igwiza uburemere bujyanye nubunini bumwe. Ibikorwa byombi birahagarika, hasigara icyitegererezo gisohoka kidahindutse, ariko ubu tensors zombi zicaye murwego rwinshuti. Igisubizo ni W8A8 (uburemere bwa 8-bit na activation) hamwe no gutakaza hafi ya zeru no gutakaza hafi 2x kwihuta no kuzigama kwibuka.

Gushyira mu bikorwa Isi

Gukora 70B-parameter LLM kuri W8A8 kuri GPU nkeya mugabanya kabiri kwibuka na matrix-kugwiza igiciro

Gushoboza INT8 umwanzuro kuri NVIDIA Hopper / Blackwell tensor cores kavukire yihutisha imibare 8-biti imibare

Kohereza moderi yo kuganira kumurongo wigiciro cyigabanije aho ibicuruzwa byikubye kabiri bigabanya byimazeyo fagitire

Gucomeka kodegisi ya transformateur kumvugo cyangwa kubisobanuro aho intoki 8-biti yihuta kandi ikonje

Ejo hazaza ha SmoothQuant na Quantisation yibikorwa

SmoothQuant yashyizeho ko ibikorwa byo kwimuka byimuka aho kwirindwa, kandi icyo gitekerezo gishimangira umusaruro INT8 na FP8. Tegereza koroshya guhuzwa hamwe na gahunda nziza cyane nka buri tsinda rigereranya, wize gupima, hamwe nubushakashatsi bwa 4-bit (urugero: uburyo bwo kumenya neza). Mugihe ibyuma bya FP8 (Hopper, Blackwell) bimaze gukura, kuringaniza-uburyo bwo kuringaniza bizakomeza gutekwa mumashanyarazi hamwe na moteri ya moteri kugirango kubara bigume hafi yubusa.

Kubishyira mubikorwa

Impamvu ari ngombwa

  • Igiciro na bije. Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
  • Ibyemezo bisobanutse. Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
  • Kugenzura ubuziranenge. Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.

Witondere

  • Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
  • Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
  • Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.

Intambwe ikurikira

  1. Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
  2. Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
  3. Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
  4. Tegura inzira yo gusubiza ibyabaye mbere yo gupima.

Komeza Ubushakashatsi

Ibaruwa y'amakuru y'ubuntu

Ibuka amakuru ya buri munsi ya AI

Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.

Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.

Isuzume

Take the SmoothQuant and Activation Quantization quiz

Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.

Tangira ikibazo

Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano

Ibibazo bikunze kubazwa

What is SmoothQuant and Activation Quantization?

SmoothQuant nubuhanga butuma bishoboka guhuza imiterere yururimi runini kugeza kuri 8-bit integer kuburemere ndetse no gukora utabanje kwitoza. Nibyingenzi kuko ibikorwa muri moderi nini zirimo gusohoka cyane mubisanzwe bisenya imibare mike-idasobanutse, kandi SmoothQuant irabahindura.

Ni ikihe kibazo SmoothQuant ikemura byumwihariko muburyo buke?

SmoothQuant yibasiye indangagaciro nini zo hanze zigaragara mumiyoboro imwe yo gukora, ubundi isenya ubunyangamugayo mugihe ibikorwa bigereranijwe kuri 8 bit.

Nigute SmoothQuant yorohereza ibikorwa byoroshye kubara?

Igabanya imiyoboro ya enterineti ku gipimo cya buri muyoboro kandi igwiza uburemere buhuye nicyo gipimo, bityo ibicuruzwa ntibihinduka ariko tensor zombi zoroha kubara.

Inyandiko W8A8 isobanura iki?

W8A8 yerekana umubare wa biti 8 kubipimo byombi (W) no gukora (A), ubutegetsi SmoothQuant butuma habaho gutakaza neza.

Kuki igipimo cya SmoothQuant cyongeraho mubyukuri nta gihe cyo gukora hejuru?

Umunzani woroshye wiziritse muburemere bwibice byabanjirije cyangwa op yahujwe mbere yigihe, kubwibyo rero nta kubara kwinyongera bibaho.

Ni uruhe ruhare alpha hyperparameter igira muri SmoothQuant?

Alpha (mubisanzwe 0.5) iringaniza ibintu byoroshya, ihitamo ingano yingorabahizi yo kwimurwa ikuwe mubikorwa no ku buremere.