LoRA na Parameter-Guhuza neza
LoRA igufasha guhitamo icyitegererezo kinini cyateguwe mugutoza agace gato k'ibiro bishya aho kuba miliyari zose.
Incamake
It is the trick that makes fine-tuning affordable on a single GPU and lets one base model serve dozens of specialized tasks.
Kwibira cyane
Byuzuye neza-kuvugurura neza uburemere bwose muburyo bw'icyitegererezo, ibyo kuri miriyari-miriyari-y'urusobe rusaba kwibuka cyane no kubika kuri buri gikorwa gishya. LoRA (Adaptation yo mu rwego rwo hasi) ifata inzira irushijeho kuba nziza: ikonjesha uburemere bwumwimerere rwose kandi igashyiramo matrike ntoya, yatozwa 'adapter' iruhande rwabo. Urufunguzo rwibanze ni uko impinduka zikenewe kugirango zimenyekanishe icyitegererezo ni urwego rwo hasi - rushobora gufatwa na matrices ebyiri zinanutse zifite ibicuruzwa bisa na matrike nini, ariko hamwe numubare muto cyane wo kwiga. Akenshi utoza munsi ya 1% yibipimo. Igisubizo ni dosiye ntoya ya adapter (rimwe na rimwe megabayiti nkeya) ushobora guhinduranya no hanze. QLoRA iragenda igereranya igipimo cyahagaritswe kugeza kuri 4-bit, bigatuma abantu bahuza neza moderi nini kubikoresho byabaguzi.
Ubushishozi
Kuri matrix yuburemere W, LoRA yerekana ivugurura ryayo nkibicuruzwa bya matrica ebyiri zo mu rwego rwo hasi, B inshuro A, aho A na B bafite igipimo gito cyimbere r (urwego, akenshi 8 cyangwa 16). Mugihe cy'amahugurwa gusa A na B bariga; W iguma ikonje. Mugusuzumisha adaptori yasohotse yongewe kumurongo wambere wibisohoka, kandi ikintu gipima (alpha) kigenzura ingaruka zacyo. Kuberako B inshuro A ishobora guhuzwa gusubira muri W nyuma yimyitozo, LoRA yongeramo zeru zidatinze zimaze guhuzwa muburyo bwoherejwe.
Ingaruka z'Ingamba
Umuvuduko n'igipimo
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika.
Kugera no kugera
Yagura uburyo bwindimi nuburyo bwo gutumanaho.
Ibyemezo bisobanutse
Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo.
Kazoza ka LoRA na Parameter-Gukora neza
Kuringaniza ibipimo byahindutse uburyo busanzwe amashyirahamwe ahuza imiterere ifunguye, kandi bizagenda byimbitse. Tegereza urusobe rwibidukikije aho amajana n'amajana ya LoRAs ashyushye cyangwa akanashyirwa hejuru yumushinga umwe usangiwe, wongeyeho sisitemu yo guhitamo itora adapteri ibereye kubisabwa. QLoRA-yuburyo bugereranijwe ikomeza gusunika ingano yicyitegererezo abakunda ibintu bashobora kwihitiramo murugo. Ubushakashatsi burakomeza muburyo bwiza bwo gutangiza, guhitamo urwego rufite imbaraga, no gukorera adapteri icyarimwe icyarimwe - gukora icyitegererezo cyumupaka fatizo umusingi wibihe byinshi bihendutse, byihariye.
Gushyira mu bikorwa Isi
Kuringaniza neza moderi ifunguye nka Llama ku bitaro by’ibitaro ukoresheje GPU imwe aho kuba cluster yuzuye
Kohereza adaptate ya MB 10 ya LoRA ihindura chatbot rusange nkumufasha winyandiko-yemewe utagabanije moderi yose
Koresha QLoRA kugirango uhuze neza moderi nini ku ikarita yerekana ibishushanyo mbonera mugupima uburemere bwibanze bwahagaritswe kugeza kuri 4-bit
Kwakira icyitegererezo kimwe shingiro hamwe no gushyushya-adaptate zitandukanye za LoRA kuri buri mukiriya kugirango akorere abafasha benshi kabuhariwe.
Ingaruka & Kurinda
Ibintu bifatika bishobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi.
Kwihuta byihuse birashobora gukora ibisubizo bidahuye mubisabwa bisa.
Ibyanditswe byumvikana birashobora kugaragara niba kugenzura kugenzura ari ntege.
Igishushanyo mbonera
Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira.
Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro.
Komeza kugenzura abantu kugenzura ibisubizo byinshi.
Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the LoRA and Parameter-Efficient Tuning quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Guhuza amabwiriza
Ibibazo bikunze kubazwa
What is LoRA and Parameter-Efficient Tuning?
LoRA igufasha guhitamo icyitegererezo kinini cyateguwe mugutoza agace gato k'ibiro bishya aho kuba miliyari zose. Nuburiganya butuma kuringaniza neza bihendutse kuri GPU imwe kandi ikareka icyitegererezo cyibanze gikora imirimo myinshi yihariye.
Ni ikihe gitekerezo cyibanze inyuma ya LoRA neza?
LoRA ituma ibiro byateganijwe bikonjeshwa kandi ikiga matrices ntoya yo hasi yongeweho hamwe nayo, itoza agace gato k'ibipimo.
Kuki LoRA igabanya cyane ikiguzi cyo gutunganya neza?
Kuberako matrike ntoya gusa irashobora gutozwa, kwibuka no kubika ibisabwa bigabanuka cyane ugereranije no kuvugurura miliyari zose zuburemere.
Niki urwego 'r' mugenzuzi wa LoRA?
Urwego r ni urwego ruto rw'imbere rusangiwe na matrices A na B; urwego r rwo hejuru rutanga adapteri ubushobozi bwinshi ariko ibipimo byinshi byo guhugura.
Nigute QLoRA yagura uburyo bwibanze bwa LoRA?
QLoRA ibika uburemere bwibanze bwahagaritswe muburyo bwa 4-bito, igabanya cyane kwibuka kuburyo moderi nini cyane zishobora guhuzwa neza numuguzi umwe GPU.
Ni ukubera iki adapteri ya LoRA yongeyeho nta yandi mananiza yatinze?
Adapter ivugurura inshuro B inshuro A ifite ishusho imwe nuburemere bwumwimerere, bityo irashobora guhunikwa neza muri W, hasigara moderi yoherejwe ingana nubunini.