Modeling làkk buñ maske
Modeling lakk buñ maskeer dafay jàngal IA bi mu mëna joxe ay baat yuñ nëbb ci anam wu jaar yoon, di jëfandikoo muy wër lépp, cammoy ak ndeyjoor yépp.
Résumé
It's the training trick behind BERT and the reason models can deeply understand sentence meaning rather than just predict what comes next.
Plongeur bu xóot
In masked language modeling (MLM), you take a sentence, randomly hide about 15% of its tokens with a special [MASK] symbol, and train the model to guess the originals. Because the model sees words on both sides of each blank, it builds a bidirectional understanding of context. BERT, introduced by Google in 2018, popularized this. A clever detail: of the masked positions, roughly 80% become [MASK], 10% are swapped for a random word, and 10% are left unchanged. This prevents the model from only ever expecting a [MASK] token at prediction time and forces robustness. After this pretraining, the model is fine-tuned for tasks like classification, question answering, and named-entity recognition.
Gis-gis xarala
MLM dafay jëfandikoo encoder Transformer ak bàyyi xel ci boppam ci ñaari yoon, kon token bu nekk dafay bàyyi xel ci ñeneen ñi benn yoon. Perte bi ñu ngi koy xayma ci position maskeer yi jëfandikoo cross-entropy ci ID token dëgg yi. Ndax bàyyi xel amul sabab (amul maskeer ëlëg), representation bu baat bu nekk dafay boole contexte cammoy ak ndijoor ci benn vecteur dense. Boobu bidirectionality mooy li model token yi ci topp bàyyi ngir mëna defar.
njeextalu pexe
Gaawaay ak yaatuwaay
Liggéeyukaay yi ci làkk yi mën nañu gëna gaaw te duñu yàq deggoo gi.
Dugg ak yegg
Dafay yaatal jëfandikoo gi ci làkk yi ak ci anam yi ñuy jokkoo.
dogal yu gëna leer
Ekip yi mën nañu gëna yàgg ci àtte ci jamono ji otomatisation di liggéey ci baamtu.
Ëlëgu modeling làkk buñ maskeer
MLM bu sell bi, xeetu dekodeer yu defar ngir chatbots yi ñoo ko ëpp doole, waaye mingi wéy di ëpp doole ci embeddings, retrieval ak classification fu xam-xam bi gëna am doole ci generation. Variante yu melni RoBERTa, ELECTRA's wecci-token gis, ak DeBERTa wéy di puus njub ak njariñ. Xaarandi encoder yu nuru MLM ñu des ci diggu seetlu, nuru semantik, ak nekk komponent yu woyof ci biir sistem yu gëna mag yu yokk ci seetlu ak multimodal fu xam-xam bu gaaw, xóot gëna am solo ci bind bu amul benn xët.
Doxal ci àdduna dëgg
Xam Google Seetug xam-xam bu sukkandiko ci BERT ci laaji waxtaan ngir delloosi xët yu gëna am njariñ.
Defar ay santaane frase ngir seetlu semantik ak sistemu seet këyit.
Defar BERT bu baax ngir jàngat yëg-yëg ci jàngat produit yi wala këyit yiy jàppale.
Njàngalem entite bu am tur buy dindi nit ñi, mbootaay yi ak bis yi ci mbindu yoon wala pajum.
Risk yi ak balustrade yi
Lépp lu jaarul yoon mën na dugg ci rapoor yi, jàppale ci liggéey bi, wala ci njariñu gëstu bi.
Sensibilite bu gaaw mën na jur njariñ yu wuute ci laajte yu noonu mel.
Done yu am solo mën nañu feeñ sudee seytu jëfandikoo gi néew doole.
Roadmap ngir samp gi
Mandargal formaa génne gi, melokaan bi, ak standard kalite yi laata ngay dugal ko.
Tontu yu am solo ak balluwaay yu wóor saa yu dëggu bi di am solo.
Fexeel am barabu xool nit ñi ngir am njariñ yu am solo.
Toppal anami gacce yi ak di faral di tàggataat ay laaj wala def-liggéey.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Masked Language Modeling quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Modeling làkk
Laaj yi ñuy faral di laaj
What is Masked Language Modeling?
Modelu làkk buñ maskeer dafay jàngal IA bi mu mëna joxe ay baat yuñ nëbb ci anam wu jaar yoon, di jëfandikoo muy wër lépp, cammoy ak ndeyjoor yépp. Mooy xeetu tàggat bi ci ginaaw BERT ak sabab bi tax model yi mëna xam bu baax li frase bi di tekki, duñu wax rek li ci topp.
Lan mooy liggéeyu tàggat yaram bu mag bi ci modeling làkk buñ maskeer?
MLM dafay nëbb benn wàll ci jeton yi ba noppi di tàggat model bi ngir ñu defaraat leen ci jëfandikoo contexte cammoy ak ndijoor yépp.
Lu tollu ci ban pursàntaasu jeton la BERT di faral di maskeer ci diiru tàggat yaram?
BERT dafay maskeer lu tollu ci 15% ci jeton yiñ dugal, muy ekilibre ci diggante joxe siñaal bu doy ak bàyyi contexte bu doy.
Lu tax MLM di joxe xeetu xam-xam bu am ñaari yoon?
Encoder Transformer dafay jëfandikoo xel mu amul sabab, suko defee token bu nekk mën gis ñeneen ñi ci ñaari wetu yi.
Ci xeetu maskeer bu BERT, lan mooy dall lu tollu ci 10% ci liggéey yiñ tànn ludul nekk [MASK]?
Ngir gëna dëgër, 10% ci position maske yi dañu leen di wecci ak benn token bu bari te 10% duñu soppiku.
Ci ban position lañuy xayma perte MLM bi?
Cross-entropy perte dañu koy jëfandikoo ci position maskeer yi, méngale waxtaan yi ak ID token yi njëkk.