Incamake
GPTQ na AWQ nuburyo bubiri buyobora bwo kugabanya imiterere yindimi zimaze gutozwa kugeza kuri 4-bit neza kuburyo zikoreshwa kubikoresho bihendutse, bito. Ninimpamvu ushobora gukoresha moderi ishoboye kumuguzi umwe GPU aho gukoresha datacenter rack.
GPTQ na AWQ Nyuma yo Guhugura Kwiga ni tekiniki yubaka tekinike igira ingaruka kumiterere yicyitegererezo, igiciro cyibikorwa remezo, ubukererwe, no kwizerwa mubipimo.
Kwibira cyane
Umubare wanyuma wamahugurwa (PTQ) uhagarika moderi yarangiye utongeye kuyitoza, ugashushanya uburemere buringaniye bugera kuri 4 kugirango ugere hafi kimwe cya kane cyo kwibuka. Ikibazo ni ugukora ibi nta gusenya neza. GPTQ (itunganijwe rya OBQ) igereranya ibipimo byapimye kurwego, ikoresheje amakuru ya kabiri yoherejwe kuva dataset ntoya kugirango ihindure uburemere busigaye kandi yishyure buri kosa ryizengurutse. AWQ (Gukora-Kumenya Ibiro Bifite uburemere) ifata indi ngingo: ireba ko agace gato k'imiyoboro yuburemere ari ingenzi cyane, igaragazwa no kureba ubunini bwibikorwa, kandi ikarinda iyo miyoboro yingenzi mugupima aho kuyibara bikabije. Byombi reka moderi nka Llama ikore muri 4-bit, nibikoresho nka vLLM, llama.cpp, na AutoGPTQ byabagize inzira nyamukuru yo gufata ibyemezo byaho kandi bikoresha neza.
Ubushishozi
GPTQ ikoresha igereranya rya Hessian (kugabanuka kw'igihombo) kugirango ihitemo uburyo kuzenguruka ibiro bimwe bigomba guhisha izindi, bikagabanya amakosa yatangijwe. AWQ isimbuka Hessians rwose: ibara ikintu kuri buri muyoboro wapima kugirango imiyoboro yingenzi yuburemere igumane neza neza, hanyuma igereranye kimwe. Byombi bigumane ibikorwa muburyo bunoze kandi bigabanya gusa uburemere, kubera ko uburemere bwiganje mububiko mugihe kugereranya ibikorwa bikunda kubabaza neza.
Kumenya GPTQ na AWQ Nyuma yo Guhugura
GPTQ na AWQ nuburyo bubiri buyobora bwo kugabanya imiterere yindimi zimaze gutozwa kugeza kuri 4-bit neza kuburyo zikoreshwa kubikoresho bihendutse, bito. Ninimpamvu ushobora gukoresha moderi ishoboye kumuguzi umwe GPU aho gukoresha datacenter rack. GPTQ na AWQ Nyuma yo Guhugura Kwiga ni tekiniki yubaka tekinike igira ingaruka kumiterere yicyitegererezo, igiciro cyibikorwa remezo, ubukererwe, no kwizerwa mubipimo. Kugirango wubake byimbitse, fata GPTQ na AWQ Nyuma yo Guhugura Kwiga nkicyitegererezo cyibikorwa, ntabwo ari ikintu kimwe: gusobanura ibyagezweho, gusobanura ibitekerezo, no gutandukanya ibyo sisitemu ishobora gukora byizewe nibisaba guca imanza zinzobere.
Mubimenyerezo, amakipe akomeye akoresha GPTQ na AWQ Nyuma yo Guhugura Amahugurwa ahindura imyubakire, amakuru, hamwe nibikorwa remezo birwanya kwizerwa nigiciro. Bandika ibipimo ngenderwaho byerekana intsinzi, bagerageza kurwanya amakuru afatika hamwe nakazi keza, kandi bagasubiramo bashingiye kubikorwa byagaragaye ko batsinzwe aho gutsinda inshuro imwe. Aha niho imyumvire yubumenyi ihinduka mubushobozi burambye kubicuruzwa, politiki, nibikorwa.
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka. Mugihe kimwe, Guhindura igipimo kimwe gishobora guhisha intege nke za sisitemu. Uburyo bukomeye cyane ni uguhuza umuvuduko wikigereranyo hamwe na disipuline yimiyoborere: kuyobora abaderevu, gufata ibimenyetso, gutangaza ibyemezo byicyemezo, no gukomeza kuvugurura uburyo bwo kwirinda nkimyitwarire yicyitegererezo, ibyo abakoresha bategereje, nibisabwa n'amategeko bigenda bihinduka.
Ingaruka z'Ingamba
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka. Mubikorwa byujuje ubuziranenge, ibi bihindurwa mumategeko agenga imikorere, imipaka nyirubwite, hamwe n'imihango yo gusubiramo kenshi kugirango amakipe ashobore kwigirira ikizere aho gupima ibidasobanutse.
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa. Mubikorwa byujuje ubuziranenge, ibi bihindurwa mumategeko agenga imikorere, imipaka nyirubwite, hamwe n'imihango yo gusubiramo kenshi kugirango amakipe ashobore kwigirira ikizere aho gupima ibidasobanutse.
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa. Mubikorwa byujuje ubuziranenge, ibi bihindurwa mumategeko agenga imikorere, imipaka nyirubwite, hamwe n'imihango yo gusubiramo kenshi kugirango amakipe ashobore kwigirira ikizere aho gupima ibidasobanutse.
Gushyira mu bikorwa Isi
Gukoresha miliyari 70-ibipimo bya Llama moderi imwe ya 24 GB y'abaguzi GPU ukoresheje uburemere bwa 4-bit GPTQ.
Moderi yuzuye ya AWQ yatanzwe kumurongo mwinshi muri vLLM kugirango umusaruro ukorwe neza APIs.
llama.cpp ukoresheje uburemere bwa GGUF kugirango ukoreshe imiterere yururimi kuri mudasobwa igendanwa.
Guhobera Isomero rya AutoGPTQ na AutoAWQ isomero ryemerera abitezimbere kugereranya moderi yakuwe mumirongo mike ya code.
Uburyo bwo Gushyira mu bikorwa
GPTQ na AWQ Kwiga nyuma yimyitozo mubikorwa
Gukoresha miliyari 70-ibipimo bya Llama moderi imwe ya 24 GB y'abaguzi GPU ukoresheje uburemere bwa 4-bit GPTQ.
Gukoresha miliyari 70-yerekana urugero rwa Llama kumurongo umwe wa 24 GB ukoresha GPU ukoresheje uburemere bwa 4 biti ya GPTQ Amakipe ubusanzwe abona ibisubizo byiza iyo asobanuye ibipimo byujuje ubuziranenge imbere, agakomeza inzira yo kuzamura abantu kubibazo, kandi akurikirana inyungu zibyara umusaruro hamwe nibiciro byamakosa mugihe runaka.
GPTQ na AWQ Kwiga nyuma yimyitozo mubikorwa
Moderi yuzuye ya AWQ yatanzwe kumurongo mwinshi muri vLLM kugirango umusaruro ukorwe neza APIs.
Moderi yagereranijwe na AWQ yatanzwe cyane muri vLLM kugirango umusaruro uhembwa neza APIs Amakipe ubusanzwe abona umusaruro ushimishije mugihe asobanura ibipimo ngenderwaho byimbere, agakomeza inzira yo kuzamura abantu kubibazo, kandi akurikirana inyungu zibyara umusaruro nibiciro byamakosa mugihe.
GPTQ na AWQ Kwiga nyuma yimyitozo mubikorwa
llama.cpp ukoresheje uburemere bwa GGUF kugirango ukoreshe imiterere yururimi kuri mudasobwa igendanwa.
llama.
GPTQ na AWQ Kwiga nyuma yimyitozo mubikorwa
Guhobera Isomero rya AutoGPTQ na AutoAWQ isomero ryemerera abitezimbere kugereranya moderi yakuwe mumirongo mike ya code.
Guhobera Isomero rya AutoGPTQ hamwe na AutoAWQ isomero ryemerera abitezimbere kugereranya urugero rwakuweho mumirongo mike ya code Amakipe akunze kubona ibisubizo byiza mugihe asobanuye ibipimo ngenderwaho byimbere, agakomeza inzira yo kuzamura abantu kubibazo, kandi akurikirana inyungu zibyara umusaruro hamwe nibiciro byamakosa mugihe runaka.
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.