Incamake
SmoothQuant nubuhanga butuma bishoboka guhuza imiterere yururimi runini kugeza kuri 8-bit integer kuburemere ndetse no gukora utabanje kwitoza. Nibyingenzi kuko ibikorwa muri moderi nini zirimo gusohoka cyane mubisanzwe bisenya imibare mike-idasobanutse, kandi SmoothQuant irabahindura.
SmoothQuant na Activation Quantisation ni inyubako ya tekiniki yubaka igira ingaruka kumiterere yicyitegererezo, igiciro cyibikorwa remezo, ubukererwe, no kwizerwa mubipimo.
Kwibira cyane
Iyo ugabanije icyitegererezo kuva 16-bit kureremba kugeza kuri 8-bit integer, uburemere buragabanuka byoroshye ariko ibikorwa nibibazo: imiyoboro imwe itwara indangagaciro zikubye inshuro 10 kugeza 100 kurenza izindi, hanyuma ukayihatira muri gride yuzuye integer yangiza ukuri. SmoothQuant, yatangijwe na Xiao n'abandi. muri 2022, abona ko uburemere bworoshye kandi bworoshye kubara mugihe ibikorwa ari spiky. Imibare rero yimura ingorane: igabanya imiyoboro yo gukora kumurongo wa buri muyoboro kandi igwiza uburemere bujyanye nubunini bumwe. Ibikorwa byombi birahagarikwa, hasigara icyitegererezo gisohoka kidahindutse, ariko ubu tensors zombi zicaye murwego rwinshuti. Igisubizo ni W8A8 (uburemere bwa 8-bit na activation) hamwe no gutakaza hafi ya zeru no gutakaza hafi 2x kwihuta no kuzigama kwibuka.
Ubushishozi
Amayeri yibanze ni kuri buri muyoboro woroshye ibintu bibarwa nka s = max (| X |) ^ alpha / max (| W |) ^ (1-alfa). Ibikorwa bipimwa na 1 / s nuburemere na s, bityo ibicuruzwa bya matrix XW birabitswe. Kuberako igipimo cyinjijwe kumurongo muburemere bwibice byabanjirije cyangwa ibikorwa byahujwe, byongera igiciro cya zeru. Alpha hyperparameter (akenshi 0.5) igenzura uburyo umutwaro wo hanze uva mubikorwa ugana ibiro.
Kumenya neza SmoothQuant na Quantisation yo gukora
SmoothQuant nubuhanga butuma bishoboka guhuza imiterere yururimi runini kugeza kuri 8-bit integer kuburemere ndetse no gukora utabanje kwitoza. Nibyingenzi kuko ibikorwa muri moderi nini zirimo gusohoka cyane mubisanzwe bisenya imibare mike-idasobanutse, kandi SmoothQuant irabahindura. SmoothQuant na Activation Quantisation ni inyubako ya tekiniki yubaka igira ingaruka kumiterere yicyitegererezo, igiciro cyibikorwa remezo, ubukererwe, no kwizerwa mubipimo. Kugirango wubake byimbitse, fata SmoothQuant na Activation Quantisation nkicyitegererezo gikora, ntabwo ari ikintu kimwe: gusobanura ibyagezweho, gusobanura ibitekerezo, no gutandukanya ibyo sisitemu ishobora gukora byizewe nibisaba guca imanza zinzobere.
Mu myitozo, amakipe akomeye akoresha SmoothQuant na Quantisation ya Activation ahindura imyubakire, amakuru, hamwe nibikorwa remezo birwanya kwizerwa nigiciro. Bandika ibipimo ngenderwaho byerekana intsinzi, bagerageza kurwanya amakuru afatika hamwe nakazi keza, kandi bagasubiramo bashingiye kubikorwa byagaragaye ko batsinzwe aho gutsinda inshuro imwe. Aha niho imyumvire yubumenyi ihinduka mubushobozi burambye kubicuruzwa, politiki, nibikorwa.
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka. Mugihe kimwe, Guhindura igipimo kimwe gishobora guhisha intege nke za sisitemu. Uburyo bukomeye cyane ni uguhuza umuvuduko wikigereranyo hamwe na disipuline yimiyoborere: kuyobora abaderevu, gufata ibimenyetso, gutangaza ibyemezo byicyemezo, no gukomeza kuvugurura uburyo bwo kwirinda nkimyitwarire yicyitegererezo, ibyo abakoresha bategereje, nibisabwa n'amategeko bigenda bihinduka.
Ingaruka z'Ingamba
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka. Mubikorwa byujuje ubuziranenge, ibi bihindurwa mumategeko agenga imikorere, imipaka nyirubwite, hamwe n'imihango yo gusubiramo kenshi kugirango amakipe ashobore kwigirira ikizere aho gupima ibidasobanutse.
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa. Mubikorwa byujuje ubuziranenge, ibi bihindurwa mumategeko agenga imikorere, imipaka nyirubwite, hamwe n'imihango yo gusubiramo kenshi kugirango amakipe ashobore kwigirira ikizere aho gupima ibidasobanutse.
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa. Mubikorwa byujuje ubuziranenge, ibi bihindurwa mumategeko agenga imikorere, imipaka nyirubwite, hamwe n'imihango yo gusubiramo kenshi kugirango amakipe ashobore kwigirira ikizere aho gupima ibidasobanutse.
Gushyira mu bikorwa-Isi
Gukora 70B-parameter LLM kuri W8A8 kuri GPU nkeya mugabanya kabiri kwibuka na matrix-kugwiza igiciro
Gushoboza INT8 umwanzuro kuri NVIDIA Hopper / Blackwell tensor cores kavukire yihutisha imibare 8-biti imibare
Kohereza moderi yo kuganira kumurongo wigiciro cyigabanije aho ibicuruzwa byikubye kabiri bigabanya byimazeyo fagitire
Gucomeka kodegisi ya transformateur kumvugo cyangwa kubisobanuro aho intoki 8-biti yihuta kandi ikonje
Uburyo bwo Gushyira mu bikorwa
SmoothQuant na Quantisation yibikorwa mubikorwa
Gukora 70B-parameter LLM kuri W8A8 kuri GPU nkeya mugabanya kabiri kwibuka hamwe na matrix-kugwiza igiciro.
Gukorera 70B-ibipimo bya LLM kuri W8A8 kuri GPU nkeya mugabanya kabiri kwibuka hamwe na matrix-kugwiza ibiciro Amakipe ubusanzwe abona ibisubizo byiza iyo asobanuye ibipimo ngenderwaho byimbere, agakomeza inzira yo kuzamura abantu kubibazo, kandi akurikirana inyungu zibyara umusaruro nibiciro byamakosa mugihe runaka.
SmoothQuant na Quantisation yibikorwa mubikorwa
Gushoboza INT8 umwanzuro kuri NVIDIA Hopper / Blackwell tensor cores kavukire yihuta 8-biti imibare.
Gushoboza INT8 umwanzuro kuri NVIDIA Hopper / Blackwell tensor cores yihuta kavukire yihuta 8-biti yimibare yimibare Amakipe ubusanzwe abona ibisubizo byiza iyo asobanuye ibipimo byujuje ubuziranenge imbere, agakomeza inzira yo kuzamura abantu kubibazo byimbitse, kandi akurikirana inyungu zibyara umusaruro nibiciro byigihe.
SmoothQuant na Quantisation yibikorwa mubikorwa
Kohereza moderi yo kuganira kumurongo wigiciro cyigabanije aho ibicuruzwa byikubye kabiri bigabanya byimazeyo fagitire.
Kohereza uburyo bwo kuganiriraho kubicuruzwa bitarangizwa nigiciro aho ibicuruzwa byikubye kabiri bigabanya mu buryo butaziguye kuri fagitire yamakipe Amakipe ubusanzwe abona ibisubizo byiza iyo asobanuye ibipimo byujuje ubuziranenge imbere, agakomeza inzira yo kuzamura abantu kubibazo, kandi akurikirana inyungu zibyara umusaruro hamwe nibiciro byamakosa mugihe runaka.
SmoothQuant na Quantisation yibikorwa mubikorwa
Gucomeka kodegisi ya transformateur kumvugo cyangwa kubisobanuro aho intoki 8-biti yihuta kandi ikonje.
Gucomeka kodegisi ya transformateur kumvugo cyangwa kubisobanuro aho ibikoresho biti 8 biti byihuta kandi bikonje Amakipe akunze kubona ibisubizo byiza iyo asobanuye ibipimo byimbere imbere, agakomeza inzira yo kuzamura abantu kubibazo, kandi akurikirana inyungu zibyara umusaruro hamwe nibiciro byamakosa mugihe runaka.
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.