Umubare
Quantisation igabanya moderi ya AI ibika nimero zayo neza, kuburyo moderi yari ikeneye data-center GPU irashobora rimwe na rimwe gukora kuri mudasobwa igendanwa cyangwa terefone.
Incamake
It is the main trick that makes large language models cheap and fast enough to deploy widely.
Kwibira cyane
Umuyoboro udasanzwe ni ikirundo kinini cyimibare bita uburemere, mubisanzwe ubikwa nka 16- cyangwa 32-bit-kureremba-ngingo. Quantisation yongeye kubika ibyo bipimo ukoresheje bits nkeya, mubisanzwe 8-bit (INT8) cyangwa na 4-bit integer. Uhereye kuri 16-bit ukagera kuri 4-biti bigabanya kwibuka hafi inshuro enye, bityo moderi ya miliyari 70-ibipimo bikenera hafi 140GB kuri 16-bit irashobora guhuza hafi 35GB kuri 4-bit. Imibare mito nayo inyura mububiko bwihuse, mubisanzwe byihutisha ibisekuruza. Ifatwa nukuri: gukanda indangagaciro zingirakamaro murwego ruto zitangiza ikosa. Uburyo bwiza bugabanya icyo gihombo uhitamo witonze ibintu bipima no kurinda uburemere bworoshye, bityo moderi yitwara hafi kimwe mugihe ikoresha igice cyibikoresho.
Ubushishozi
Buri tsinda ryibipimo bibona igipimo cyerekana ikarita nyayo kumurongo muto wuzuye; kugwiza inyuma kubipimo hafi yubaka umubare wambere. Uburyo bwo kubara nyuma yimyitozo nka GPTQ na AWQ isesengura dataset ntoya kugirango hamenyekane uburemere bwingenzi kandi ushireho umunzani kugirango ugabanye amakosa yasohotse, aho kuzenguruka ibintu byose buhumyi. Ibikorwa bikunze kubikwa neza cyane kuko biratandukanye mugihe cyo gukora. Igisubizo nicyitegererezo kibika 4-bit integer ariko ikabara ibisubizo hafi cyane ya verisiyo yuzuye.
Ingaruka z'Ingamba
Umuvuduko n'igipimo
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika.
Kugera no kugera
Yagura uburyo bwindimi nuburyo bwo gutumanaho.
Ibyemezo bisobanutse
Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo.
Kazoza ka Quantisation
Tegereza kubara guhinduka nkibisanzwe aho kuba optimiz. Abacuruzi b'ibyuma barimo kongeramo kavukire 4-bit, ndetse no munsi ya bito-bito, hamwe na tekinike nko kumenya umubare-wamahugurwa-yamenyesheje guteka kwihanganira ibintu bike mubyitegererezo kuva bigitangira, bikagabanya igihombo cyukuri kurushaho. Ubushakashatsi mubice 2-bit na 1-bit (binary) byerekana birakora, bigamije gukora moderi zishoboye kuri terefone na chip yashyizwemo. Mugihe ku bikoresho na AI byigenga bikura, moderi zingana neza zizaba ingenzi kubafasha bungirije ahohereza amakuru kubicu.
Gushyira mu bikorwa Isi
Gukoresha uburyo bwo kuganira nka Llama mugace ka GPU ukoresha umuguzi wa 4-bit ya GGUF cyangwa GPTQ aho gukenera amakarita menshi yamakuru.
Abafasha ku bikoresho kuri terefone, aho moderi 8-bit cyangwa 4-biti reka imvugo nibiranga inyandiko bikore nta murongo uhuza.
Kugabanya ibicu byerekana ibiciro kubakiriya-baterankunga ukoresheje moderi ya INT8, ihuza ibyifuzo byinshi kuri buri GPU.
Ibikoresho byo kumpande nka kamera yubwenge cyangwa sensor ya IoT ikoresha comptabilite igereranya iyerekwa-imvugo yerekana imipaka ntarengwa.
Ingaruka & Kurinda
Ibintu bifatika bishobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi.
Kwihuta byihuse birashobora gukora ibisubizo bidahuye mubisabwa bisa.
Ibyanditswe byumvikana birashobora kugaragara niba kugenzura kugenzura ari ntege.
Igishushanyo mbonera
Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira.
Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro.
Komeza kugenzura abantu kugenzura ibisubizo byinshi.
Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Quantization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Ibisigisigi bya Vector
Ibibazo bikunze kubazwa
What is Quantization?
Quantisation igabanya moderi ya AI ibika nimero zayo neza, kuburyo moderi yari ikeneye data-center GPU irashobora rimwe na rimwe gukora kuri mudasobwa igendanwa cyangwa terefone. Nuburiganya nyamukuru butuma imiterere yururimi runini ihendutse kandi byihuse kugirango ikoreshwe henshi.
Ni ubuhe buryo bwo kubara buhindura cyane cyane urusobe rw'imitsi?
Quantisation yongeye kubika uburemere bwikitegererezo muburyo bwo kubara neza, nka 8-bit cyangwa 4-bit integer aho kuba 16- cyangwa 32-bit.
Hafi ya bangahe yibukwa mukwimura ibiro kuva 16-bit kugeza kuri 4-bit?
4 bits ni kimwe cya kane cya 16 bits, kubwibyo kubika ibiro bigabanuka hafi kimwe cya kane, kugabanuka kwa 4x.
Ni izihe ngaruka nyamukuru zo kugereranya ubukana?
Guhagararira indangagaciro hamwe ninzego nke zitangiza kuzenguruka, bishobora gutesha agaciro ubuziranenge niba bidacunzwe neza.
Ni ubuhe buryo nka GPTQ na AWQ bukoresha dataset ntoya?
Ubu buryo nyuma yamahugurwa asesengura ibyitegererezo bike kugirango ashyireho ibintu bipima kandi arinde uburemere bworoshye, agumane ibisubizo hafi yumwimerere.
Kuki kubara akenshi bituma icyitegererezo cyihuta, atari gito gusa?
Indangagaciro-zifatika zifata ububiko buke bwibikoresho byo kwikorera no kwimuka, akenshi bikaba icyuho mugihe cyibisekuru, bityo umwanzuro wihuta.