Cakudar Masana LoRA
Cakudar Kwararrun LoRA (MoLE) ya haɗu da ƙananan adaftan da aka horar da su da arha tare da na'ura mai ba da hanya tsakanin hanyoyin sadarwa don haka ƙirar tushe guda ɗaya na iya ƙware a sassa daban-daban na ayyuka, salo, ko ƙwarewa.
Dubawa
It matters because it brings the modularity of Mixture-of-Experts to fine-tuning without retraining huge networks.
Zurfafa nutsewa
LoRA (Ƙarancin Matsayi) yana daskare ma'aunin ƙirar ƙirar da aka riga aka horar kuma yana horar da ƙananan ƙananan matrices waɗanda ke lalata halayen sa, suna yin gyara mai arha. Cakuɗewar Ƙwararrun LoRA suna horar da nau'ikan adaftan da yawa, kowannensu yana ɗaukar fasaha daban-daban, yanki, ko ra'ayi na gani, sannan yana ƙara ƙaramar hanyar sadarwar gating wacce ke yanke shawarar waɗanne adaftan don kunna (da yadda ƙarfi) don shigarwar da aka bayar. Maimakon sauti mai kyau guda ɗaya, kuna samun ɗakin karatu na ƙwararrun masana. Mai amfani da na'ura mai ba da hanya tsakanin hanyoyin sadarwa na iya haɗa ƙwararru a kowane Layer da kowace alama, don haka tambayar coding na iya ja adaftar Python yayin da labari ya ba da labari. Wannan yana guje wa tsangwama da bala'in mantawa da wannan annoba horar da adaftan guda ɗaya akan ayyuka masu gauraye da yawa a lokaci ɗaya, kuma yana ƙyale ƙungiyoyi su ƙara ko cire ƙwarewa ba tare da taɓa kashin baya ba.
Fahimtar Fasaha
Kowane ƙwararren LoRA yana shigar da delta W = B * A, inda A da B ƙananan matrices ne (mutumin sau da yawa 4-64). Aikin gating yana samar da ma'auni akan ƙwararru, kuma ana haɗa abubuwan da aka fitar azaman jimlar ma'auni (mai laushi mai laushi) ko zaɓi na sama-k (madaidaicin kewayawa). Mahimmanci ma'aunin tushe yana tsayawa a daskare, don haka adaftar da na'ura mai ba da hanya tsakanin hanyoyin sadarwa kawai aka horar da su. A cikin nau'ikan hoto masu yaduwa, gating na matsayi yana koyan ma'auni na kowane Layer don haka ra'ayi da yawa LoRAs ya tsara ba tare da wani ya rinjayi sauran ba.
Dabarun Tasiri
Kudin da kasafin kuɗi
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Shawarwari masu haske
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Kula da inganci
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Makomar Cakudawar Masana LoRA
Yi tsammanin kasuwannin adaftan inda samfura ke ɗaukar ƙwararrun LoRA na al'umma akan buƙata, da na'urori masu amfani da hanya waɗanda ke gano ta atomatik waɗanda ƙwararrun aikin ke buƙata a lokacin tantancewa. Bincike yana turawa zuwa ga abun da aka koya wanda ke warware rikice-rikice tsakanin adaftan, babban rabo ga kowane ƙwararru, da haɗa MoLE tare da ƙaramin ƙirar ƙirar MoE don ƙwarewa na mataki biyu. Kan na'ura da tura kayan aiki sun fi amfana sosai, tunda musanya adaftan megabyte kaɗan ya fi arha fiye da jigilar sabbin samfura.
Aiwatar da Gaskiyar Duniya
Mataimakin lambar da ke tafiya tsakanin ƙwararrun LoRA daban-daban don Python, SQL, da Tsatsa ya danganta da fayil ko faɗakarwa, guje wa tsoma baki cikin yare.
Masu amfani da Stable Diffusion suna tattara ɗabi'a da salon LoRAs tare da shingen gating don haka hoto yana kiyaye takamaiman fuska da salon fasaha ba tare da launi ko dalla-dalla ba.
Kamfanin chatbot na kasuwanci yana loda adaftan kowane bangare (doka, HR, kudi) akan ƙirar tushe iri ɗaya daskararre, yana musanya su ba tare da sake aiki ba.
Samfurin tallafi na harsuna da yawa tare da ƙwararren LoRA ɗaya kowane harshe, wanda aka gano ta hanyar shigar da yaren da aka gano don kiyaye ƙwarewar kowane harshe.
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Mixture of LoRA Experts quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Cakudawar Masana
Tambayoyin da ake yawan yi
What is Mixture of LoRA Experts?
Cakudar Kwararrun LoRA (MoLE) ya haɗu da ƙananan adaftan da aka horar da su da arha tare da na'ura mai ba da hanya tsakanin hanyoyin sadarwa don haka ƙirar tushe guda ɗaya na iya ƙware a sassa daban-daban na ayyuka, salo, ko ƙwarewa. Yana da mahimmanci saboda yana kawo yanayin gaurayawa-na-ƙwararru don daidaitawa ba tare da sake horar da manyan hanyoyin sadarwa ba.
Menene sashin 'LoRA' na Cakudar Masana LoRA ke nufi?
LoRA yana tsaye ga Ƙarƙashin Matsayi: yana daskare samfurin tushe kuma yana horar da ƙananan matakan matrix waɗanda ke daidaita ɗabi'a cikin rahusa.
Menene aikin gating/na'ura mai ba da hanya tsakanin hanyoyin sadarwa a MoLE?
Na'ura mai ba da hanya tsakanin hanyoyin sadarwa tana samar da ma'auni akan ƙwararrun LoRA da ake da su, zaɓi da haɗa su ta kowace shigarwa (kuma sau da yawa kowane Layer ko alama).
Me yasa MoLE sau da yawa yafi kyau fiye da horar da adaftan akan ayyuka masu gauraye da yawa?
Kwararrun masana daban-daban suna guje wa babban mantawa da tsangwama na ɗawainiya waɗanda ke faruwa lokacin da adaftan ɗaya dole ne ya koyi ƙwarewar saɓani da yawa lokaci guda.
Yayin horon MoLE, menene gaba ɗaya ke faruwa da ma'aunin da aka riga aka horar da ƙirar?
Kashin baya ya kasance a daskare; ƙananan ƙwararrun LoRA kawai da cibiyar sadarwar gating suna karɓar sabuntawar gradient.
A cikin nau'ikan hoto masu yaduwa, wace matsala ce gating na matsayi/per-layer a cikin MoLE ke taimakawa wajen warwarewa?
Per-Layer gating yana koyon yadda ƙarfin kowane adaftan ke ba da gudummawa a kowane Layer, yana barin ra'ayi da yawa LoRAs su haɗu ba tare da rinjaye ɗaya ba.