Tuning prefix
Tuning prefix anam bu baax la ci parametre ngir méngale modelu làkk bu liw ci di tàggat ay vecteur yu ndaw yuy wéy di am prepended ci bépp dugal bu layer.
Résumé
It lets you customize giant models for new tasks while updating less than 1% of parameters.
Plongeur bu xóot
Prefix tuning, bi gëstukat yu Stanford Li ak Liang dugal ci 2021, dafay méngale transformatër buñ tàggat bu njëkk te du laal diisaayam. Duñu defar parametre yépp, waaye dafay njëkka def ay 'jeton virtuel' yuñ mëna tàggat (prefix bi) ci caabi yi ak valeur yi ci bepp couche attention. Modèle frozen dafay bàyyi xel ci prefix bi melni muy contexte dëgg, di yóbbu doxalinam ci liggéey biñ bëgga def. Ndax vecteur prefix yi kese lañuy jàng, mën nga denc benn prefix bu ndaw ci liggéey bu nekk ci barabu kopi model bu mat. Loolu dafay tax liggéey yu bari yomb ba noppi moytu dencukaay bi gëna bari ay fine-tuning. Dafay def lu baax ci liggéey yu melni table-to-text ak resumé, di faral di méngoo ak ajustement bu mat sëkk ci jekkal done yu bari.
Gis-gis xarala
Tuning prompt wuute na ak vecteur yi ci couche embedding bi, tuning prefix dafay dugal vecteur key/valeur yuñ mëna tàggat ci bepp couche transformateur. Ngir dakkal tàggat bi, prefix bi dafay gëna bari ci ab reso feed-forward bu ndaw (ab reparameterization trick) moo gën ñu ko defar ci saasi; reso boobu dañu koy sànni ginaaw biñu tàggatee, bàyyi matris prefix yiñ jàng rek. Yii parametru prefix kese ñooy am gradient yi—yax gi yépp dafay des ci liw.
njeextalu pexe
Gaawaay ak yaatuwaay
Liggéeyukaay yi ci làkk yi mën nañu gëna gaaw te duñu yàq deggoo gi.
Dugg ak yegg
Dafay yaatal jëfandikoo gi ci làkk yi ak ci anam yi ñuy jokkoo.
dogal yu gëna leer
Ekip yi mën nañu gëna yàgg ci àtte ci jamono ji otomatisation di liggéey ci baamtu.
Ëlëgu Tuning Prefix
Tuning prefix jàppale na ñu tàmbali vague tuning bu baax (PEFT) te mingi wéy di nekk bloku tabax ci bibliotek yu melni Hugging Face PEFT. Ginaaw modelu base yi dañuy gëna màgg ba yegg ci téemeeri miliyaar ciy parametre, adaptatër yu woyof yi melni prefix yi dañuy gëna xëcc nit ñi ci serwiis yu bari ak personalisasioŋ ci aparey bi. Xaarandi wéyal jegewaale hybrid yiy jaxasoo ak prefix ak LoRA-style low-rank yeesali, ak màgg jëfandikoo ci di yoree stil, persona, ak jeffin kaaraange te duñu tàggataat model yépp.
Doxal ci àdduna dëgg
Defar benn yaxu ndigg bu GPT-2 ngir defar tablo-ci-tekst ci tàggat ab prefix bu ndaw ci setu done WebNLG
Dafay joxe ay fukki-fukki xeeti tënk yuñ jagleel kiliyaan yi ci benn model buñ bokk, bu nekk ci ñoom ni fichier prefix buñ mëna wecci
Digal ab ton wala persona modelu lakk ngir ab chatbot te doo taggaat poid yu njëkk yi
Adaptaasioŋ domen bu am done yu néew, lu ci melni defar mbindu yoon wala pajum, fu ajustement bu mat sëkk mën na ëpp
Risk yi ak balustrade yi
Lépp lu jaarul yoon mën na dugg ci rapoor yi, jàppale ci liggéey bi, wala ci njariñu gëstu bi.
Sensibilite bu gaaw mën na jur njariñ yu wuute ci laajte yu noonu mel.
Done yu am solo mën nañu feeñ sudee seytu jëfandikoo gi néew doole.
Roadmap ngir samp gi
Mandargal formaa génne gi, melokaan bi, ak standard kalite yi laata ngay dugal ko.
Tontu yu am solo ak balluwaay yu wóor saa yu dëggu bi di am solo.
Fexeel am barabu xool nit ñi ngir am njariñ yu am solo.
Toppal anami gacce yi ak di faral di tàggataat ay laaj wala def-liggéey.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Prefix Tuning quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Réjection de échantillonnage de rejection
Laaj yi ñuy faral di laaj
What is Prefix Tuning?
Tuning prefix anam bu baax la ci parametre ngir méngale modelu làkk bu liw ci di tàggat ay vecteur yu ndaw yuy wéy di am prepended ci bépp dugal bu layer. Daf lay may nga mëna tànn sa bopp ay model yu mag ngir liggéey yu bees boole ci yeesal lu néew 1% ci paramet yi.
Lan mooy prefix tuning?
Tuning prefix dafay gelé yaxu ndigg li, te du jàng ludul ay vecteur prefix yu wéy di wéy, suko defee mu gëna am doole ci parametre yi.
Fan lañu dugal vecteur prefix yi?
Prefix tuning dafay njëkka tàggat vecteur clé/valeur ci bépp couche transformateur buy bàyyi xel ci boppam, du duggal bi kese.
Kan moo dugal prefix tuning ak kañ la?
Xiang Lisa Li ak Percy Liang ñoo ko tàmbale ci Stanford ci 2021.
Lan moo waral prefix tuning neex ngir liggéey lu bari?
Ndax prefix bu ndaw bi kese mooy liggéey bu nekk, dangay denc benn fichier bu ndaw ci liggéey bu nekk moo gën ngay duppi model bi yépp.
Ban pexe lañuy faral di jëfandikoo ngir dakkal tàggat prefix bi?
Benn MLP bu ndaw dafay defar prefix bi ci diiru tàggat ngir stabilite, ginaaw ga ñu sànni ko, bàyyi matrices prefix yiñ jàng rek.