Ubuyobozi bwa tekiniki

GPTQ na AWQ Nyuma yo Guhugura

GPTQ na AWQ nuburyo bubiri buyoboye bwo kugabanya imiterere yindimi zimaze gutozwa kugeza kuri 4-bitike neza kuburyo ikora kubikoresho bihendutse, bito.

Umva iki gitaboAI-byavuzwe · ~2 min

Kuriyi page2 min soma
  1. Incamake
  2. Kwibira cyane
  3. Gushyira mu bikorwa Isi
  4. Kazoza ka GPTQ na AWQ Nyuma yo Guhugura
  5. Kubishyira mubikorwa
  6. Komeza Ubushakashatsi
  7. Ibibazo bikunze kubazwa

Incamake

Ninimpamvu ushobora gukoresha moderi ishoboye kumuguzi umwe GPU aho kuba datacenter rack.

Kwibira cyane

Umubare wanyuma wamahugurwa (PTQ) uhagarika moderi yarangiye utongeye kuyitoza, ugashushanya uburemere buringaniye bugera kuri 4 kugirango ugere hafi kimwe cya kane cyo kwibuka. Ikibazo ni ugukora ibi nta gusenya neza. GPTQ (itunganijwe rya OBQ) igereranya ibipimo byapimye kurwego, ikoresheje amakuru ya kabiri yoherejwe kuva dataset ntoya kugirango ihindure uburemere busigaye kandi yishyure buri kosa ryizengurutse. AWQ (Gukora-Kumenya Ibiro Bifite uburemere) ifata indi ngingo: ireba ko agace gato k'imiyoboro yuburemere ari ingenzi cyane, igaragazwa no kureba ubunini bwibikorwa, kandi ikarinda iyo miyoboro yingenzi mugupima aho kuyibara bikabije. Byombi reka moderi nka Llama ikore muri 4-bit, nibikoresho nka vLLM, llama.cpp, na AutoGPTQ byabagize inzira nyamukuru yo gufata ibyemezo byaho kandi bikoresha neza.

Gushyira mu bikorwa Isi

Gukoresha miliyari 70-ibipimo bya Llama kumurongo umwe wa 24 GB ukoresha GPU ukoresheje uburemere bwa 4-bit GPTQ.

Moderi yuzuye ya AWQ yatanzwe kumurongo mwinshi muri vLLM kugirango umusaruro ukorwe neza APIs.

llama.cpp ukoresheje uburemere bwa GGUF kugirango ukoreshe imiterere yururimi kuri mudasobwa igendanwa.

Guhobera Isomero rya AutoGPTQ na AutoAWQ isomero ryemerera abitezimbere kugereranya moderi yakuwe mumirongo mike ya code.

Kazoza ka GPTQ na AWQ Nyuma yo Guhugura

Quantisation iri gusunika munsi ya 4 bits yerekeza kuri 3-bit, 2-bit, hamwe na gahunda ivanze-yuzuye, akenshi ihujwe nubusa. Tegereza guhuza hamwe na moteri ikora kugirango igabanye, KV-cache compression, hamwe na decoding yibikorwa hamwe. Inkunga yibikoresho kumiterere-bito nka NVFP4 na MXFP4 irakura, kandi ibikoresho byikora bizagenda bitoranya ubugari bwa biti. Intego yagutse ni hafi-gutakaza 4-bit (na hepfo) nkibisanzwe, gukora moderi zikomeye zihendutse gukorera ahantu hose.

Kubishyira mubikorwa

Impamvu ari ngombwa

  • Igiciro na bije. Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
  • Ibyemezo bisobanutse. Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
  • Kugenzura ubuziranenge. Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.

Witondere

  • Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
  • Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
  • Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.

Intambwe ikurikira

  1. Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
  2. Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
  3. Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
  4. Tegura inzira yo gusubiza ibyabaye mbere yo gupima.

Komeza Ubushakashatsi

Ibaruwa y'amakuru y'ubuntu

Ibuka amakuru ya buri munsi ya AI

Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.

Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.

Isuzume

Take the GPTQ and AWQ Post-Training Quantization quiz

Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.

Tangira ikibazo

Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano

Ibibazo bikunze kubazwa

Niki GPTQ na AWQ Nyuma yo Guhugura?

GPTQ na AWQ nuburyo bubiri buyobora bwo kugabanya imiterere yindimi zimaze gutozwa kugeza kuri 4-bit neza kuburyo zikoreshwa kubikoresho bihendutse, bito. Ninimpamvu ushobora gukoresha moderi ishoboye kumuguzi umwe GPU aho gukoresha datacenter rack.

Bisobanura iki 'kugereranya nyuma y'amahugurwa' bisobanura?

Umubare wanyuma wamahugurwa ugabanya neza uburemere bwikitegererezo cyarangiye (urugero, kugeza kuri 4 bits) utarinze kubisubiramo.

Ni ayahe makuru GPTQ ikoresha kugirango yishyure amakosa yo kuzenguruka mugihe cyo kubara?

GPTQ ikoresha Hessian igereranya kugirango yumve uburyo kubara ibiro bimwe bigira ingaruka kubihombo, hanyuma igahindura ibiro bisigaye kugirango yishyure.

Ni ubuhe bushishozi bwingenzi butwara AWQ (Gukora-Kumenya Ibiro)?

AWQ igaragaza imiyoboro ihambaye ikoresheje ubunini bwa activation kandi ikabarinda ikoresheje igipimo, kubera ko imiyoboro mike ifite akamaro kanini.

Hafi yangahe yibuka bingana na 4-bit yo kubika hamwe nuburemere bwa 16-bit?

Uhereye kuri 16 bits ukagera kuri 4 kuri buri buremere bigabanya uburemere bwibiro bigera kuri kimwe cya kane, hafi ya 4x kugabanuka.

Ni ukubera iki GPTQ na AWQ byombi bigereranya uburemere ariko bikagumya gukora neza?

Ibiro nibyo byingenzi byibukwa kwibuka, mugihe ibikorwa byunvikana cyane no gutakaza neza, kubwibyo uburemere-gusa nibisanzwe biryoshye.