LoRA da Daidaita-Ingantacciyar Tunawa
LoRA yana ba ku damar keɓance ƙaton ƙirar ƙirar da aka riga aka horar ta hanyar horar da ƙaramin saitin sabbin ma'auni maimakon duk biliyoyin.
Dubawa
It is the trick that makes fine-tuning affordable on a single GPU and lets one base model serve dozens of specialized tasks.
Zurfafa nutsewa
Cikakken daidaitawa yana sabunta kowane nauyi a cikin samfuri, wanda don hanyar sadarwa ta siga biliyoyin da yawa yana buƙatar babban ƙwaƙwalwar ajiya da ajiya ga kowane sabon ɗawainiya. LoRA (Ƙaramar-Rank Adafta) yana ɗaukar hanya mafi wayo: yana daskare ainihin ma'aunin nauyi gaba ɗaya kuma yana sanya ƙananan matrices 'adapter' masu horarwa tare da su. Makullin fare shine canjin da ake buƙata don ƙware ƙirar ƙira ba shi da daraja - ana iya kama shi ta hanyar matrices masu fata guda biyu waɗanda samfurin su daidai yake da matrix mai nauyi, amma tare da ƙarancin lambobi don koyo. Yawancin lokaci kuna horar da ƙasa da 1% na sigogi. Sakamakon shine ƙaramin fayil ɗin adaftar (wani lokaci ƴan megabytes) zaku iya musanya ciki da waje. QLoRA ya ci gaba ta hanyar ƙididdige tushen daskararre zuwa 4-bit, yana barin mutane su daidaita manyan samfuran akan kayan masarufi.
Fahimtar Fasaha
Don matrix W mai nauyi, LoRA yana wakiltar sabuntawar sa azaman samfurin ƙananan matrices biyu, B sau A, inda A da B suna da ƙaramin girman ciki r (matsayi, sau da yawa 8 ko 16). A lokacin horo kawai A da B ake koyan; W yana daskarewa. Bisa la'akari da fitarwar adaftar ana ƙarawa zuwa ainihin abin fitarwa na Layer, kuma ma'aunin sikeli (alpha) yana sarrafa tasirinsa. Saboda lokutan B A ana iya haɗa su cikin W bayan horo, LoRA ta ƙara ƙarin jinkiri sau ɗaya an haɗa shi cikin ƙirar da aka tura.
Dabarun Tasiri
Gudu da sikelin
Gudun aikin harshe na iya tafiya da sauri ba tare da sadaukar da daidaito ba.
Shiga ku isa
Yana faɗaɗa damar shiga cikin harsuna da salon sadarwa.
Shawarwari masu haske
Ƙungiyoyi za su iya ciyar da ƙarin lokaci akan hukunci yayin da aiki da kai ke sarrafa maimaitawa.
Makomar LoRA da Ingantaccen Daidaitawa
Daidaita ingantaccen siga ya zama tsohuwar hanyar ƙungiyoyin daidaita buɗaɗɗen ƙira, kuma hakan zai zurfafa. Yi tsammanin yanayin yanayin adaftar inda ɗaruruwan LoRAs ke da zafi-swapped ko ma an haɗa su a saman tushe ɗaya ɗaya, da tsarin kewayawa waɗanda ke ɗaukar adaftan da ya dace akan buƙatu. QLoRA mai ƙididdigewa mai ƙididdigewa yana ci gaba da tura girman samfuran da masu sha'awar sha'awa za su iya keɓancewa a gida. An ci gaba da bincike kan mafi kyawun farawa, zaɓin matsayi mai ƙarfi, da kuma hidimar adaftar adaftan da yawa lokaci guda yadda ya kamata - yin ƙirar tushe ɗaya na kan iyaka don arha da yawa, bambance-bambance na musamman.
Aiwatar da Gaskiyar Duniya
Kyakkyawan daidaita samfurin buɗe kamar Llama akan bayanan asibiti na asibiti ta amfani da GPU guda ɗaya maimakon cikakken tari.
Aika adaftar LoRA 10 MB wanda ke juya babban chatbot zuwa mataimaki na takaddun doka ba tare da sake rarraba dukkan samfurin ba.
Amfani da QLoRA don daidaita babban samfuri akan katin zane na mabukaci ta hanyar ƙididdige ma'aunin tushe mai daskararre zuwa 4-bit
Bayar da samfurin tushe guda ɗaya da canza canjin LoRA daban-daban akan kowane abokin ciniki don hidimar mataimaka na musamman da yawa cikin rahusa
Hatsari & Tsare-tsare
Abubuwan da aka ruɗe suna iya shigar da rahotanni cikin nutsuwa, kwararar tallafi, ko abubuwan bincike.
Hankali na gaggawa na iya ƙirƙirar sakamako mara daidaituwa a cikin buƙatun iri ɗaya.
Za a iya fallasa bayanan rubutu mai ma'ana idan ikon samun dama yana da rauni.
Taswirar Hanya
Ƙayyade tsarin fitarwa, sautin, da ma'auni masu inganci kafin fitowa.
Amsa a ƙasa tare da amintattun tushe a duk lokacin da daidaito ya shafi mahimmanci.
Ajiye wurin binciken ɗan adam don abubuwan da ake samu masu girma.
Bibiyar tsarin gazawar kuma sake horar da tsokaci ko tafiyar aiki akai-akai.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the LoRA and Parameter-Efficient Tuning quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Tunatarwa Umarni
Tambayoyin da ake yawan yi
What is LoRA and Parameter-Efficient Tuning?
LoRA yana ba ku damar keɓance ƙaton ƙirar ƙirar da aka riga aka horar ta hanyar horar da ƙaramin saitin sabbin ma'auni maimakon duk biliyoyin. Dabarar ce ke ba da araha mai araha akan GPU ɗaya kuma yana barin ƙirar tushe guda ɗaya ta yi ayyuka da yawa na musamman.
Menene ainihin ra'ayin da ke bayan daidaitawar LoRA?
LoRA tana kiyaye ma'aunin da aka riga aka horar da su a daskare kuma yana koyon ƙananan matakan matrices da aka haɗa tare da su, yana horar da ɗan ƙaramin juzu'i ne kawai.
Me yasa LoRA ke rage tsadar farashin daidaitawa?
Saboda ƙananan matrix adaftar kaɗai ke iya horarwa, ƙwaƙwalwar ajiya da buƙatun ajiya sun ragu sosai idan aka kwatanta da sabunta duk biliyoyin ma'auni.
Menene darajar 'r' a cikin adaftar LoRA?
Matsayin r shine ƙaramin girman ciki wanda matrices A da B ke raba; r mafi girma yana ba adaftar ƙarin ƙarfi amma ƙarin sigogi don horarwa.
Ta yaya QLoRA ke tsawaita ainihin hanyar LoRA?
QLoRA yana adana ma'aunin tushe mai daskararre a daidaitaccen 4-bit, yana yanke ƙwaƙwalwar ajiya sosai don haka manyan samfuran za'a iya daidaita su akan GPU mai amfani guda ɗaya.
Me yasa adaftar LoRA da aka haɗa ta ke ƙara rashin jinkiri?
Sabunta adaftan B sau A yana da siffar daidai da nauyin asali, don haka ana iya naɗe shi kai tsaye zuwa W, yana barin ƙirar da aka tura daidai girman da sauri.