Jàng ci biir muy tënk
Jàng ci biir kontekst mooy mën mën bu yéeme bi modeli làkk yu mag yi am ngir jël liggéey bu bees ci misaal yu néew yuñ def ci prompt bi, te kenn du leen tàggataat.
Résumé
It is the reason you can 'teach' a model on the fly just by showing it what you want.
Plongeur bu xóot
Daanaka jàngal ab reso neuronal liggéey bu bees mooy yeesal ay poid ci tàggat yaram. Jàng ci biir contexte wuute na ak yeneen yi: dangay bind ay misaal yu néew ci prompt bi ('contexte' bi), model bi dafay jël motif bi ba noppi jëfandikoo ko ci dugal bu bees. Amul lenn luy soppiku ci biir model bi; misaal yi dañuy tegtal token bi ci topp. Di nga dégg 'zero-shot' (tegtal kese), 'one-shot' (benn misaal), ak 'few-shot' (misal yu bari). GPT-3 moo siiwal doxalin boobu ci 2020 ba noppi mujjee nekk mën-mën buy feeñ: model yu ndaw mënu ñu ko def, waaye ginaaw lu tollu ci 100-milyaar-parametre, njub ci yenn laaj-tiire dafay yéeg bu baax. Royuwaay bi jàng na bu baax xàmmee ak wéyal motif yi ci diiru tàggat-yaram, suko defee mu mëna jëfandikoowaat mën-mën boobu ci diiru inference.
Gis-gis xarala
Gëstub tekki-tekki dafa topp lu bari ci mën-mën bii ci 'boppu induksioŋ' - sircuit yuy bàyyi xel ci yiy feeñ ci diiru tàggat-yaram ba noppi def prefix-matching bu fësul: ñuy scanner dellu ngir gis fi benn token bu noonu mel feeñ, ba noppi ñu kopie li ci topp. Kon sudee sa laaj dafa wane 'pomme -> fruit, karoot -> lujum', model bi dafay méngoo ak jumtukaay bi ba noppi di wax etiketu mbir mi ci topp. Li gëna am solo mooy amul benn gradient buy daw ak amul benn yeesal poid ci inference bi. misaal yi dañuy soppi jëmmu aktivaasioŋ yiy dundal seddaleb probabilite token bi ci topp.
njeextalu pexe
Gaawaay ak yaatuwaay
Liggéeyukaay yi ci làkk yi mën nañu gëna gaaw te duñu yàq deggoo gi.
Dugg ak yegg
Dafay yaatal jëfandikoo gi ci làkk yi ak ci anam yi ñuy jokkoo.
dogal yu gëna leer
Ekip yi mën nañu gëna yàgg ci àtte ci jamono ji otomatisation di liggéey ci baamtu.
Ëlëgu jàng ci biir contexte
Yaatal palanteer contexte (leegi téemeeri junni token) push in-context jàng ci regime 'many-shot', fu fukki-fukki wala téemeeri misaal mëna xëcc fine-tuning ngir yenn liggéey, te amul benn njëgu tàggat. Xaarandil lëkkaloo bu gëna dëgër ak seetlu, suko defee misaal yu am solo yi ñuy jëlee ci saasi, ak theorie bu gëna baax ci su jàng ci biir kontekst bi lajj wala di distract. Dina des nekk anam wu gaaw, yomb ngir méngale benn model, di mottali - du wecci - fine-tuning ngir liggéey yu stabil, yu bari ay volume.
Doxal ci àdduna dëgg
Jox benn chatbot ñetti misaali këyit yuy jàppale ak seeni kategori, ba noppi nga fexe bamu xaaj këyit bu bees ci anam woowu
Dafay wane xeetu ñaari mbir balaa/ginnaaw ñaari mbind yu jaxasoo yuñ formaatewaat ci JSON bu sell suko defee mu soppi leneen li
Teg ñaari misaali tegtali produit ci melokaanu sa màrk suko defee yeneen yu bees yi méngoo ak stil bi
Dafay wane ab jafe-jafe math bu jafee xam, ñu liggéey ko jéego ci jéego suko defee model bi mëna saafara jafe-jafe yu noonu mel ak benn xeetu xalaat
Risk yi ak balustrade yi
Lépp lu jaarul yoon mën na dugg ci rapoor yi, jàppale ci liggéey bi, wala ci njariñu gëstu bi.
Sensibilite bu gaaw mën na jur njariñ yu wuute ci laajte yu noonu mel.
Done yu am solo mën nañu feeñ sudee seytu jëfandikoo gi néew doole.
Roadmap ngir samp gi
Mandargal formaa génne gi, melokaan bi, ak standard kalite yi laata ngay dugal ko.
Tontu yu am solo ak balluwaay yu wóor saa yu dëggu bi di am solo.
Fexeel am barabu xool nit ñi ngir am njariñ yu am solo.
Toppal anami gacce yi ak di faral di tàggataat ay laaj wala def-liggéey.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the In-Context Learning quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Windows Kontekst
Laaj yi ñuy faral di laaj
What is In-Context Learning?
Jàng ci biir kontekst mooy mën mën bu yéeme bi modeli làkk yu mag yi am ngir jël liggéey bu bees ci misaal yu néew yuñ def ci prompt bi, te kenn du leen tàggataat. Moo tax nga mëna 'jàngale' model ci saasi soo ko wan li nga bëgg.
Lan mooy soppeeku ci biir model bi ci diiru jàng ci biir contexte?
Jàng ci biir contexte mingi aju ci inference. misaal yi ci prompt bi dañuy forme activation model bi ak next-token predictions, waaye amul benn poid buñu yeesal.
Lan moo waral ñuy wax ni jàng ci biir contexte mën nañu ko wax ni 'mën-mën buy feeñ'?
Mën-mën bi dafa néew doole wala amul ci model yu ndaw yi, te dafay yéeg bu baax su model yi yeggee ci escale bu rëy lool, moo tax ñu koy woowe emergent.
Ban mekaniism bi ci biir moo gëna méngoo ak jàng ci biir transformatër yi?
Liggéeyu tekki xamme boppu induction - sircuit attention yiy gis fi token bu noonu mel feeñe ak koppi li ci topp - muy dawalkat bu mag ci jàng ci contexte.