Ururimi AI
QLoRA na 4-Bit Byiza-Guhuza
QLoRA nubuhanga butuma uhuza neza urugero rwururimi runini kumuguzi umwe GPU ubika moderi yahagaritswe mubice 4 gusa kuburemere.
Umva iki gitaboAI-byavuzwe · ~2 min
Kuriyi page2 min soma
Incamake
Cyakoze gukora moderi ya 65B-parameter ishoboka kubikoresho byabanje gukora gusa moderi igice cyubunini.
Kwibira cyane
Mubisanzwe, gutunganya neza moderi nini bisobanura gupakira uburemere bwose muri 16-bit neza no kuvugurura byose, bisaba kwibuka cyane. QLoRA ikomatanya ibitekerezo bibiri. Ubwa mbere, ihagarika icyitegererezo cyateganijwe ikagereranya kugeza kuri 4 bits, igabanya kwibuka hafi inshuro enye. Icya kabiri, ikoresha LoRA: aho kuvugurura matrike nini yuburemere, itera inshinge ntoya yo gutozwa yo mu rwego rwo hasi ya adaptor matrices iruhande rwabo, bityo miriyoni nkeya gusa zikaba zivugururwa. 4-bit shingiro igumaho mugihe gradients zinyura gusa muri adaptate nto. Yamenyekanye mu 2023 na Dettmers na bagenzi be, QLoRA yerekanye ko guhuza neza moderi ya 65B kuri imwe ya 48GB GPU ishobora guhuza ubuziranenge bwuzuye 16-bitunganijwe neza.
Gushyira mu bikorwa Isi
Gutangira neza-gutunganya neza moderi ya 70B Llama kuri 48GB GPU imwe kugirango yubake umufasha-wunganira abakiriya mumajwi yayo bwite adakodesha cluster ya seriveri.
Umushakashatsi ufite umuguzi umwe RTX 4090 ahuza icyitegererezo gifunguye niche yubuvuzi-busubiza dataset ijoro ryose.
Iterambere rirema udukingirizo duto duto, twahinduranya LoRA adapteri kubikorwa bitandukanye, byose bigabana moderi imwe ya 4-bit fatizo yuzuye mububiko.
Hobbyist ahindura neza icyitegererezo kumurongo wibiganiro byabo kugirango bigane uburyo bwo kwandika ukoresheje ibyuma bya Colab-yubusa.
Kazoza ka QLoRA na 4-Bit Nziza-Guhuza
4-bit-neza-guhuza byahindutse imyitozo isanzwe, kandi ubushakashatsi ubu busunika kugera no kuri verisiyo yo hasi, harimo 2-bit na 1-bit (ternary). Gahunda nshya yo kubara nka AWQ, GPTQ, na HQQ inonosora neza, mugihe tekinike nka QA-LoRA igamije kugumya kwerekana urugero na nyuma yo guhuza adapt. Mugihe uburemere buremereye bugenda bwiyongera, tegereza ibikoresho bituma abashimisha bahuza neza 70B-wongeyeho moderi imwe kumikino imwe ya GPU kugirango ibe gahunda, demokarasi yihariye.
Kubishyira mubikorwa
Impamvu ari ngombwa
- Umuvuduko n'igipimo. Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika.
- Kugera no kugera. Yagura uburyo bwindimi nuburyo bwo gutumanaho.
- Ibyemezo bisobanutse. Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo.
Witondere
- Ibintu bifatika bishobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi.
- Kwihuta byihuse birashobora gukora ibisubizo bidahuye mubisabwa bisa.
- Ibyanditswe byumvikana birashobora kugaragara niba kugenzura kugenzura ari ntege.
Intambwe ikurikira
- Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira.
- Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro.
- Komeza kugenzura abantu kugenzura ibisubizo byinshi.
- Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho.
Komeza Ubushakashatsi
Ibaruwa y'amakuru y'ubuntu
Ibuka amakuru ya buri munsi ya AI
Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.
Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.
Isuzume
Take the QLoRA and 4-Bit Fine-Tuning quiz
Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.
Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano
Ibibazo bikunze kubazwa
Niki QLoRA na 4-Bit Nziza-Guhuza?
QLoRA nubuhanga butuma uhuza neza urugero rwururimi runini kumuguzi umwe GPU ubika moderi yahagaritswe mubice 4 gusa kuburemere. Cyakoze gukora moderi ya 65B-parameter ishoboka kubikoresho byabanje gukora gusa moderi igice cyubunini.
Niki gitekerezo cyibanze cyo kubika kwibuka inyuma ya '4-bit' igice cya QLoRA?
QLoRA ibika uburemere bwakonje bwateganijwe kuri bits 4 kuri buri gaciro, kugabanya kwibuka inshuro enye ugereranije na 16-bit.
Mugihe QLoRA itunganya neza, ni ibihe bipimo mubyukuri bigenda bivugururwa na gradient descente?
Icyitegererezo fatizo cyahagaritswe; gusa inshinge zo murwego rwo hasi adapter matrices yakira ivugurura rya gradient, nigice gito cyibipimo.
NF4 ni iki mu rwego rwa QLoRA?
NF4.
Ni ikihe kibazo 'paje optimizers' muri aderesi ya QLoRA?
Paged optimizers use NVIDIA unified memory to page optimizer states between GPU and CPU memory as needed; the QLoRA paper says this helps manage memory spikes and reduce out-of-memory risk during long-sequence fine-tuning.
Ni ryari ibiro 4-biti bihindurwa bisobanutse neza mugihe cy'amahugurwa?
During a matrix multiplication, quantized weights are dequantized on the fly to a higher-precision compute dtype (for example, BF16 or FP16 in QLoRA configurations); the temporary higher-precision values are not stored as a full base-model copy.
Komeza wige
Kuyobora
Abandi bayobozi batoranijwe kuriyi ngingo