Incamake
Kwigereranya kwicyitegererezo bigabanya urusobe rwimitsi ibika umubare wabyo muke, bityo moderi imwe ikora byihuse no kubikoresho bito. Ninimpamvu nyamukuru moderi nini zishobora guhura na GPU imwe, mudasobwa igendanwa, cyangwa na terefone.
Model Quantization nicyuma cyubaka tekinike kigira ingaruka kumiterere yicyitegererezo, igiciro cyibikorwa remezo, ubukererwe, no kwizerwa mubipimo.
Kwibira cyane
Moderi yatojwe mubisanzwe ibika buri buremere nka 32-bit cyangwa 16-biti ireremba-ingingo. Quantisation isimbuza abafite imiterere-yuzuye yo hasi nka 8-bit integer (INT8) cyangwa 4-biti (INT4), guca kwibuka hafi 4x kugeza 8x. Moderi ya miliyari 70-ikeneye hafi 140GB muri 16-bit irashobora kugabanuka hafi ya 35GB kuri 4-bit, bikwiranye na GPU imwe. Gufata ni ukuri: gukanda indangagaciro zitandukanye mu ndobo 256 cyangwa 16 zitakaza ibisobanuro. Uburyo bugezweho nka GPTQ, AWQ, nuburyo bwa NF4 bukoreshwa muri QLoRA butoranya ibintu bipima ubwenge kandi bikarinda uburemere bworoshye, bityo gutakaza ubuziranenge akenshi ni bito. Quantisation niyo mpamvu ibikoresho nka llama.cpp na Ollama bishobora gukoresha moderi zishoboye mugace nta data center.
Ubushishozi
Ikarita yerekana agaciro nyako kuri gride ntoya ikoresheje umunzani na zeru-point: yabitswe_int = uruziga (agaciro / igipimo) + zero_ point. Guhitamo igipimo neza ni umukino wose. Kuri buri muyoboro cyangwa kuri buri tsinda gupima umunzani utandukanye kubice bya matrix yuburemere, bikabika neza aho bifite akamaro. Umubare wanyuma wamahugurwa uhindura gusa icyitegererezo cyarangiye, mugihe imyitozo-yo kumenya kugereranya kwigana mugihe cyamahugurwa kugirango urusobe rwige kubyihanganira, mubisanzwe bitanga neza-bito neza.
Kumenya Kwerekana Icyitegererezo
Kwigereranya kwicyitegererezo bigabanya urusobe rwimitsi ibika umubare wabyo muke, bityo moderi imwe ikora byihuse no kubikoresho bito. Ninimpamvu nyamukuru moderi nini zishobora guhura na GPU imwe, mudasobwa igendanwa, cyangwa na terefone. Model Quantization nicyuma cyubaka tekinike kigira ingaruka kumiterere yicyitegererezo, igiciro cyibikorwa remezo, ubukererwe, no kwizerwa mubipimo. Kugirango wubake byimbitse, fata Quantisation yicyitegererezo nkicyitegererezo gikora, ntabwo ari ikintu kimwe: gusobanura ibyagezweho, gusobanura ibitekerezo, no gutandukanya ibyo sisitemu ishobora gukora byizewe nibisaba guca imanza zinzobere.
Mu myitozo, amakipe akomeye akoresha Model Quantisation atezimbere ubwubatsi, amakuru, hamwe nibikorwa remezo birwanya kwizerwa nigiciro. Bandika ibipimo ngenderwaho byerekana intsinzi, bagerageza kurwanya amakuru afatika hamwe nakazi keza, kandi bagasubiramo bashingiye kubikorwa byagaragaye ko batsinzwe aho gutsinda inshuro imwe. Aha niho imyumvire yubumenyi ihinduka mubushobozi burambye kubicuruzwa, politiki, nibikorwa.
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka. Mugihe kimwe, Guhindura igipimo kimwe gishobora guhisha intege nke za sisitemu. Uburyo bukomeye cyane ni uguhuza umuvuduko wikigereranyo hamwe na disipuline yimiyoborere: kuyobora abaderevu, gufata ibimenyetso, gutangaza ibyemezo byicyemezo, no gukomeza kuvugurura uburyo bwo kwirinda nkimyitwarire yicyitegererezo, ibyo abakoresha bategereje, nibisabwa n'amategeko bigenda bihinduka.
Ingaruka z'Ingamba
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka. Mubikorwa byujuje ubuziranenge, ibi bihindurwa mumategeko agenga imikorere, imipaka nyirubwite, hamwe n'imihango yo gusubiramo kenshi kugirango amakipe ashobore kwigirira ikizere aho gupima ibidasobanutse.
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa. Mubikorwa byujuje ubuziranenge, ibi bihindurwa mumategeko agenga imikorere, imipaka nyirubwite, hamwe n'imihango yo gusubiramo kenshi kugirango amakipe ashobore kwigirira ikizere aho gupima ibidasobanutse.
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa. Mubikorwa byujuje ubuziranenge, ibi bihindurwa mumategeko agenga imikorere, imipaka nyirubwite, hamwe n'imihango yo gusubiramo kenshi kugirango amakipe ashobore kwigirira ikizere aho gupima ibidasobanutse.
Gushyira mu bikorwa Isi
Gukoresha moderi ya 7B cyangwa 13B Llama kuri mudasobwa igendanwa hamwe na llama.cpp cyangwa Ollama ukoresheje dosiye ya 4-bit ya GGUF.
QLoRA gutunganya neza moderi nini kuri GPU imwe ukomeza uburemere bwibanze bukonjeshwa muri 4-bit NF4.
Kohereza moderi ya INT8 kuri terefone hamwe nigihe cyo gukora-ibikoresho kugirango abafasha bakore kumurongo kandi wenyine.
Gukora ibiciro bihendutse bya API aho INT8 / FP8 igereranya hafi inshuro ebyiri ibyinjira kandi bigabanya ikiguzi cyo kwibuka.
Uburyo bwo Gushyira mu bikorwa
Kwerekana icyitegererezo mubikorwa
Gukoresha moderi ya 7B cyangwa 13B Llama kuri mudasobwa igendanwa hamwe na llama.cpp cyangwa Ollama ukoresheje dosiye ya 4-bit ya GGUF.
Gukoresha moderi ya 7B cyangwa 13B ya Llama kuri mudasobwa igendanwa hamwe na llama.cpp cyangwa Ollama ukoresheje amadosiye 4 ya biti ya GGUF Amakipe ubusanzwe abona ibisubizo byiza iyo asobanuye ibipimo byujuje ubuziranenge imbere, agakomeza inzira yo kuzamura abantu kubibazo, kandi akurikirana inyungu zibyara umusaruro nibiciro byamakosa mugihe runaka.
Kwerekana icyitegererezo mubikorwa
QLoRA gutunganya neza moderi nini kuri GPU imwe ukomeza uburemere bwibanze bukonjeshwa muri 4-bit NF4.
Q.
Kwerekana icyitegererezo mubikorwa
Kohereza moderi ya INT8 kuri terefone hamwe nigihe cyo gukora-ibikoresho kugirango abafasha bakore kumurongo kandi wenyine.
Kohereza moderi ya INT8 kuri terefone hamwe nigihe cyo gukora ku bikoresho kugirango abafasha bakore kumurongo kandi abikorera ku giti cyabo Amakipe akunze kubona umusaruro ushimishije mugihe asobanura ibipimo ngenderwaho byimbere, agakomeza inzira yo kuzamura abantu kubibazo byimbitse, kandi agakurikirana inyungu zibyara umusaruro nibiciro byamakosa mugihe runaka.
Kwerekana icyitegererezo mubikorwa
Gukora ibiciro bihendutse bya API aho INT8 / FP8 igereranya hafi inshuro ebyiri ibyinjira kandi bigabanya ikiguzi cyo kwibuka.
Gukora amaherezo ya API ahendutse aho kubara INT8 / FP8 byikubye kabiri kwinjiza no kugabanya ikiguzi cyo kwibuka Amakipe ubusanzwe abona ibisubizo byiza iyo asobanuye ibipimo byujuje ubuziranenge imbere, agakomeza inzira yo kuzamura abantu kubibazo, kandi akurikirana inyungu zibyara umusaruro hamwe nibiciro byamakosa mugihe runaka.
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.