Tuning bu gaaw
Prompt tuning dafay méngale xeetu làkk bu gelaase ci jàng ay vecteur 'soft prompt' yu wéy yu ñu njëkka def ci input bi, moo gën ñuy bind kàddu yi ak loxo.
Résumé
It is one of the leanest ways to specialize a giant model, and it gets better as models get bigger.
Plongeur bu xóot
Akordaasioŋ bu gaaw bi gëstukat yi di Lester, Al-Rfou ak Constant dugal ci 2021, mooy cousin bi gëna yomba akordaasioŋ prefiks. Dunga defar ab laajtu bind ak sa loxo, dangay gelé model bi yépp ba noppi jàng benn matrix bu ndaw bu am ay embeddings yuy wéy—'sañu laaj yu yomb'—yi ñuy njëkka def ci couche de input bi kese. Wàcci degrade dafay méngale vecteur yii ngir ñu mëna am doxalin bu jaar yoon ci benn liggéey. Gis-gis bu yéeme: ginnàaw bi modelu base bi di yokk ba yegg ci ay miliyaar ciy parametre, ajustement bu gaaw bi dafay tëj bërëb bi ak ajustement bu mat sëkk, mujjee ko méngale ak ay benchmark yu melni SuperGLUE. Liggéey bu nekk soxlawul ludul ab soft prompt bu ndaw (dafay faral di nekk ay junni parametre yu néew), kon benn model frozen mën na def liggéey yu bari benn yoon. Auteur yi dañu ko kaadre ni 'dooley echel ngir parametre yu gaaw yi.'
Gis-gis xarala
Soft prompts duñu baat dëgg—dañuy vecteur yuy flote ci biir espace buñ samp te soxla wuñu méngoo ak benn token ci vocabulaire bi. Dañu leen di yokk ci couche biy dugal rek (di wuute ak tuning prefix, biy dugal ci couche bu nekk), muy tax tuning bi gëna woyof. Ndax model bi dafa gelé, gradient yi duñu dellu ci embedding yu soft-prompt yi kese. Tàmbali, guddaay bi gaaw, ak yaatuwaayu model bi dañu am njeexital yu am solo ci kalite bi.
njeextalu pexe
Gaawaay ak yaatuwaay
Liggéeyukaay yi ci làkk yi mën nañu gëna gaaw te duñu yàq deggoo gi.
Dugg ak yegg
Dafay yaatal jëfandikoo gi ci làkk yi ak ci anam yi ñuy jokkoo.
dogal yu gëna leer
Ekip yi mën nañu gëna yàgg ci àtte ci jamono ji otomatisation di liggéey ci baamtu.
Ëlëgu tuning bu gaaw
Tuning bu gaaw siiwal xalaat biy wax ni mën nga doxal modelu fondation bu liw ak siñaal yu ndaw yuñ jàng, te dafay jàppale lu bari ci jumtukaayi PEFT tay. Ginaaw model yi dañuy wéy di yokk, effet gap-closing bi dafay tax ay prompt yu woyof di xëcc ay liggéey yu bari yu yomb. Gëstu mingi yaatal xalaat bi ngir toxal ay laaj yuñ mëna jàng ci liggéey ak xeetu liggéey, boole leen ak seetlu, jëfandikoo leen ngir defar ay mbir yuñ mëna saytu te gëna wóor. Xaarandil ay laaj yu woyof ngir des levier bu yomb njëg ci wetu LoRA ak adaptatër yi.
Doxal ci àdduna dëgg
Benn xeetu T5 buñ jagleel ay liggéey yu bari ci SuperGLUE, denc ab laaj bu wuute ci liggéey bu nekk
Defar benn model bu rëy ci kiliyaan yu bari, ku nekk ak limu jàng
Yoriinu yëg-yëg wala jeffin ju xaaj te doo defar kàddu yi ak loxo
Toxal ci saasi bu yomb: njëkka tàggat santaane ci benn liggéey ngir tàmbali jàng ci beneen bu jëm ci
Risk yi ak balustrade yi
Lépp lu jaarul yoon mën na dugg ci rapoor yi, jàppale ci liggéey bi, wala ci njariñu gëstu bi.
Sensibilite bu gaaw mën na jur njariñ yu wuute ci laajte yu noonu mel.
Done yu am solo mën nañu feeñ sudee seytu jëfandikoo gi néew doole.
Roadmap ngir samp gi
Mandargal formaa génne gi, melokaan bi, ak standard kalite yi laata ngay dugal ko.
Tontu yu am solo ak balluwaay yu wóor saa yu dëggu bi di am solo.
Fexeel am barabu xool nit ñi ngir am njariñ yu am solo.
Toppal anami gacce yi ak di faral di tàggataat ay laaj wala def-liggéey.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Prompt Tuning quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Prompt Engineering
Laaj yi ñuy faral di laaj
What is Prompt Tuning?
Prompt tuning dafay méngale xeetu làkk bu gelaase ci jàng ay vecteur 'soft prompt' yu wéy yu ñu njëkka def ci input bi, moo gën ñuy bind kàddu yi ak loxo. Benn la ci anam yi gëna woyof ngir specialise model bu mag, te lu model yi di gëna rëy di gëna rafet.
Luy 'soft prompt' ci tuning bu gaaw?
Soft prompt yi ay vecteur yu wéy yuñ mëna tàggat lañu ci espace embedding, duñu ay baat yu nit mëna jàng.
Fan la tuning bu gaaw bi di yokk ay vecteur yuñ mëna tàggat?
Tuning prompt dafay njëkka am prompt yu woyof ci input bi, moo tax mu gëna woyof ci tuning prefix.
Ban njeexte bu yéeme la këyit bi njëkk fësal ci wàllu eskaalu model bi?
Titulu këyit bi dafa fësal 'dooley eskaal bi': model yu gëna mag yi dañuy may tuning bi gaaw mu xëcc tuning bu mat sëkk.
Kan moo dugal tuning bu gaaw?
Brian Lester, Rami Al-Rfou, and Noah Constant at Google published the prompt tuning paper in 2021.
naka lay def ba mëna def ay liggéey yu bari ci benn model?
Benn prompt soft bu ndaw kese mooy liggéey bi, kon benn model frozen mën na dalal liggéey yu bari ci weccoo prompt yi.