Modeling làkk
Modelu làkk mooy liggéey bu yomb te gëmloo nit, di wax luy waaja am ci baat bi wala token bi ci topp, buñu sukkandikoo ci mbind mi fi ñu tolli nii.
Résumé
This single objective, scaled up massively, is what produces today's powerful chatbots and writing assistants.
Plongeur bu xóot
Ci core bi, modelu làkk dafay jox probabilite yi ci toppalante mbind yi. Bu ñu sukkandikoo ci laaj bi 'Kapitaalu Faraas moy', defay xayma ni beppu token bu ci topp munna am, te 'Paris' defa wara am poñ yu bari. Royuwaayi làkk yu njëkk ya nekkoon n-gram yu lim, di lim ñaata yoon la baat yi di feeñ, waaye dañu daan sonn lool ci muy tekki lu gudd ak mbind yuñu gisul. Modèlu làkku neuronal yi ñoo wecci lim ak misaal yuñ jàng, ba noppi architecture transformateur bi tàmbalee ci 2017 may model yi ñu mëna toppatoo mbind yu gudd yi ci anam wu jaar yoon. Modèlu làkk yu mag yu bees yu melni famiy GPT dañu leen tàggat ci corpus text yu mag ak benn mébet: wax luy token bi ci topp. Li yéeme mooy, def lii bu baax dafay forse model bi mu jàng grammer bi, mbir yi am, motif yi ñuy xalaat, ak stil bi, ndax soo bëggee wax luy waaja am ci mbind mi, fàww nga xam ko bu baax. Generation dafay liggéey ci di wax luy waaja am token bi ci topp ba noppi diko joxaat.
Gis-gis xarala
Royuwaayi làkk yi gëna bari ci jamono jii dañuy dellu ginaaw seen bopp: dañuy boole probabilite yi ci benn frase ci produit bi ci probabilite yi ci topp, di wax luy waaja am benn token ci saa si cammoñ dem ndeyjoor. Taggat yaram dafay wàññi cross-entropy perte, loolu mooy neexal joxe probabilite bu rëy ci token bi ci topp ci bind tàggat yaram. Lii dafay saytu boppam, etiketu yi dañu bawoo ci mbind mi ci boppam, kon soxla wuñu benn tegtal bu nit. Ci jamonoy defar, pexe jël misaal yu melni tàngoor, top-k, ak top-p (nucleus) ñooy saytu njaay mi am ci digganté génne giñ mëna seentu ak génne giñ mëna sos.
njeextalu pexe
Gaawaay ak yaatuwaay
Liggéeyukaay yi ci làkk yi mën nañu gëna gaaw te duñu yàq deggoo gi.
Dugg ak yegg
Dafay yaatal jëfandikoo gi ci làkk yi ak ci anam yi ñuy jokkoo.
dogal yu gëna leer
Ekip yi mën nañu gëna yàgg ci àtte ci jamono ji otomatisation di liggéey ci baamtu.
Ëlëgu modeling làkk
Next-token prediction dafa am doole lool, te yooni scaling dañu wane ni model yu gëna mag ak done yu bari ñu ngi wéy di yokk kàttan, doonte benefiis yi dañuy yeex, te done yu baax dañuy gëna néew. Frontier bi mingi toxu ci xalaat, palanteer yu gëna gudd, ak pexe ginaaw tàggat yaram yu melni jàngat bu am doole ci feedback nit biy jëmmal doxalin ginaaw bi ñu tabaxee modelu base bi. Xaarandi wéyal njaxasu modelu làkk ak jumtukaay, seet, ak dugal multimodal, ci noonu la jubluwaay bu am solo bi di wax luy waaja am ci token bi ci topp mooy nekk fondaasioŋ bi leneen lépp tabax ci kaw.
Doxal ci àdduna dëgg
Defal boppam ci sa klaweeru telefon wala sa e-mail di la xelal baat bi ci topp sooy bind
Chatbot bu melni ChatGPT defar ab tontu bu neex di wax luy waaja am token bi ci topp
Editëru kode yu melni GitHub Copilot di wax luy waaja am ci ligne kode bi ci topp ci muy wër
Sistem yiy xàmmee kàddu yi di jëfandikoo xeetu làkk ngir tànn transkripsioŋ bi gëna wóor ci tànneef yiy nuru
Risk yi ak balustrade yi
Lépp lu jaarul yoon mën na dugg ci rapoor yi, jàppale ci liggéey bi, wala ci njariñu gëstu bi.
Sensibilite bu gaaw mën na jur njariñ yu wuute ci laajte yu noonu mel.
Done yu am solo mën nañu feeñ sudee seytu jëfandikoo gi néew doole.
Roadmap ngir samp gi
Mandargal formaa génne gi, melokaan bi, ak standard kalite yi laata ngay dugal ko.
Tontu yu am solo ak balluwaay yu wóor saa yu dëggu bi di am solo.
Fexeel am barabu xool nit ñi ngir am njariñ yu am solo.
Toppal anami gacce yi ak di faral di tàggataat ay laaj wala def-liggéey.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Language Modeling quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Modeling làkk buñ maske
Laaj yi ñuy faral di laaj
What is Language Modeling?
Modelu làkk mooy liggéey bu yomb te gëmloo nit, di wax luy waaja am ci baat bi wala token bi ci topp, buñu sukkandikoo ci mbind mi fi ñu tolli nii. Benn mébet boobu, ñu yokk ko bu baax, moo jur chatbot yu am doole yi ak assistant bind yu tay yi.
Lan mooy liggéey bi modelu làkk wara def?
Benn modelu làkk dafay xayma probabilite li ci topp ci benn rang, ba noppi generation dafay liggéey ci baamtu wax luy waaja am ak yokk token bi ci topp.
Lan moo nekkoon ay jafe-jafe yu mag ci xeetu làkk n-gram yu njëkk ya?
Modèle N-gram yi dañu sukkandikoo ci lim baat yu gàtt yu toppalante, moo tax dañu baña mëna jëfandikoo bu baax contexte yu sori yi, ba noppi dañu lajj ci mbind yu ñu musul gis.
Lan moo waral ñuy woowe tàggat yaram ci misaalu làkk 'di saytu sa bopp'?
Token bi ci topp mingi fi nekk ci biir mbind mi, kon model bi dafay sos ay mébetam boppam te du am benn annotation manuel.
Luy 'autorégressif' ci xeetu làkk?
Modèle autorégressive yi dañuy defar token par token, di conditioner bepp prediction bu bees ci lépp luñu defar ba leegi.
Ban fonction perte lañuy gëna jëfandikoo ngir tàggat modelu làkk?
Taggat dafay wàññi cross-entropy, di neexal xeetu joxe probabilite bu rëy ci token bi ci topp bi ñuy seetlu ci mbindu taggat bi.