Ururimi AI
Umubare
Ibyiciro by'ubugenzuzi byerekana agaciro ka neural-network ahitamo mu rwego ruto cyangwa rufite intera itandukanye y'ibipimo.
Umva iki gitaboAI-byavuzwe · ~3 min
Kuriyi page3 min soma
Incamake
Ibi bishobora kugabanya ububiko no guhindura ibisabwa mu mibare, mu gihe ingaruka ku nyandiko n'igihe bisaba uburyo n'uburyo bwo kubishyira mu bikorwa.
Kwibira cyane
A neural network uses numerical values for weights and intermediate activations. Quantization changes the representation of selected values, often reducing the number of bits used for storage. A weight-only scheme should be distinguished from one that also quantizes activations. Bit counts provide an ideal payload calculation: storing the same number of weights at four bits instead of sixteen uses one-quarter of the raw packed-weight bits. That arithmetic does not include scale metadata, unquantized values, activations, caches, or temporary workspace, so it is not a promise about total device memory. Quantization can introduce rounding or other approximation error. Its effect on predictions needs evaluation rather than an assumption that every compressed model stays identical to its baseline. Calibration procedures use representative inputs to guide decisions. GPTQ uses approximate second-order information for post-training weight quantization. AWQ uses activation observations to identify salient weight scaling choices; activation-aware describes information used for weight quantization, not a rule that all activations are themselves stored as low-bit integers. Lower-bit packing can reduce transferred bytes, but runtime gains also depend on kernels, hardware, and conversion overhead. Storage and arithmetic precision can differ: QLoRA, for example, distinguishes low-bit stored weights from higher-precision computation. An affine integer representation offers a useful illustration. With a stated reconstruction rule of scale times code minus zero point, code six, scale one-half, and zero point two reconstruct the value two. Quantization-aware training instead exposes training to the effects of the chosen quantization procedure so learned values can adapt; post-training methods act on an already trained starting model.
Gushyira mu bikorwa Isi
Gusuzuma ikigereranyo cya GPTQ cyakozwe ku bwoko bw'ibiro by'ubugenzuzi n'uburyo bw'ibanze bw'ukuri bw'ubwoko bw'ibiro bwuzuye.
Kwigira ku buryo AWQ ikoreshwa amakuru y'ibikorwa kugira ngo hitamo uburyo bwo kugabanya ibiro.
Gukura ibice by'ibiro by'ubwoko bw'ibanze ku buryo bw'ibice, ibikorwa, n'ibindi by'ububiko bw'akazi.
Gusuzuma ubugenzuzi nyuma yo gukora ibikorwa n'uburyo bwo gukora ubugenzuzi bw'ubugenzuzi.
Kazoza ka Quantisation
Uburyo bwo gukora ubugenzuzi buto bushobora kugereranywa mu buryo bw'ububiko, igihe, n'ibisabwa mu bikorwa. Inyungu y'ubushakashatsi ku murongo umwe n'igikoresho igomba gutangwa hamwe n'ibisobanuro by'ubugenzuzi n'uburyo bwo kubishyira mu bikorwa. Ibice by'ibiro by'ubugenzuzi byoroheje bishobora gufasha umushinga ufite ububiko buke, mu gihe kernels zitashyigikiwe cyangwa ibiciro byo guhindura bishobora guhindura umusaruro. Uburyo bwo gukora ubugenzuzi n'uburyo bwo gukora ubugenzuzi nyuma bifite amahitamo atandukanye yo guhindura, kandi imiterere y'ubugenzuzi bw'ubunini butandukanye ishobora gutanga ubunini bwihariye. Gusesengura ububiko bw'akazi bwuzuye n'ibikorwa by'icyifuzo aho kwemeza ubushobozi bwo gushyira mu bikorwa, ubuzima bwite, cyangwa ubunini bw'ibipimo gusa.
Kubishyira mubikorwa
Impamvu ari ngombwa
- Umuvuduko n'igipimo. Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika.
- Kugera no kugera. Yagura uburyo bwindimi nuburyo bwo gutumanaho.
- Ibyemezo bisobanutse. Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo.
Witondere
- Ibintu bifatika bishobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi.
- Kwihuta byihuse birashobora gukora ibisubizo bidahuye mubisabwa bisa.
- Ibyanditswe byumvikana birashobora kugaragara niba kugenzura kugenzura ari ntege.
Intambwe ikurikira
- Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira.
- Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro.
- Komeza kugenzura abantu kugenzura ibisubizo byinshi.
- Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho.
Komeza Ubushakashatsi
Ibaruwa y'amakuru y'ubuntu
Ibuka amakuru ya buri munsi ya AI
Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.
Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.
Isuzume
Fata ikizamini cya Quantization
Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.
Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano
Ibibazo bikunze kubazwa
Quantisation ni iki?
Ibyiciro by'ubugenzuzi byerekana agaciro ka neural-network ahitamo mu rwego ruto cyangwa rufite intera itandukanye y'ibipimo. Ibi bishobora kugabanya ububiko no guhindura ibisabwa mu mibare, mu gihe ingaruka ku nyandiko n'igihe bisaba uburyo n'uburyo bwo kubishyira mu bikorwa.
Mu buryo bwo gukora ubugenzuzi bwibanda ku bintu, ni ikihe gikorwa cy'ubugenzuzi cyahindutse?
Ibyiciro by'ubugenzuzi bihindura uburyo bwo kugaragaza imibare; andi mafaranga ashobora no kwibanda ku bikorwa.
Kwibagirwa metadata n'ibindi by'ububiko bw'akazi, ni ikihe gice cy'ibipimo by'ibiro by'ubugenzuzi byibanze rihagaze igihe uhuza ibiro by'ubugenzuzi bimwe na bimwe kuva kuri 16-bit kugeza kuri 4-bit?
Icyiciro cya kane gikoreshwa ku cyiciro cya gatandatu ni kimwe mu cyiciro; ububiko bw'igihe cyose burimo byinshi kurenza ibice by'ibiro.
Ni izihe ngaruka nyamukuru zo kugereranya ubukana?
Guhagararira indangagaciro hamwe ninzego nke zitangiza kuzenguruka, bishobora gutesha agaciro ubuziranenge niba bidacunzwe neza.
Kuki uburyo bwa GPTQ na AWQ bwerekana amakuru y'ubugenzuzi?
GPTQ na AWQ bikoresha uburyo butandukanye bwo gukurikirana amakuru y'ubugenzuzi; ibisubizo byabo bisaba igenzura mu buryo bw'ibikorwa n'uburyo.
Kuki igipimo cy'igihe cyakorwa cyakorwa aho kwemeza gusa ku bitaramo by'ibiro by'ubugenzuzi?
Gukora ibice bishobora kugabanya ibyinjira, mu gihe inzira y'ibikorwa igena niba ibyo bigira ingaruka ku muvuduko.
Komeza wige
Kuyobora
Abandi bayobozi batoranijwe kuriyi ngingo