Luqadda AI HAGAHA
Qaabaynta Luuqada Madow
Masked language modeling (MLM) trains a bidirectional encoder to predict selected input tokens from surrounding context.
Dhageyso hagahanAI-waa laga sheekeeyay · ~1 min
Boggaan1 daqiiqo akhri
Dulmar
BERT uses a token-corruption scheme; later methods alter the masking or prediction objective.
quusid qoto dheer
BERT’s pretraining combines masked language modeling with a sentence-pair objective in its original formulation. For MLM, selected tokens are hidden or altered and the model predicts the original tokens using both left and right context. RoBERTa removes the next-sentence prediction objective in its experiments and uses dynamic masking. ELECTRA instead trains a discriminator to classify original versus generator-replaced tokens. These approaches differ in objective, data handling and model design.
Dhaqangelinta Adduunka-dhabta ah
BERT masks selected token positions and computes prediction loss for those positions.
RoBERTa uses dynamic masking so selected positions can vary across passes.
ELECTRA trains a discriminator to detect tokens replaced by a generator.
Compare MLM with autoregressive next-token training, which uses a causal context.
Mustaqbalka Qaabaynta Luuqada Madow
Choose a pretraining objective based on the model’s intended use and available compute/data. Report tokenizer, masking policy and evaluation tasks. MLM’s bidirectional context supports representation learning, while autoregressive training naturally matches left-to-right generation; neither objective is universally best.
Ku dhaqangelinta
Maxay muhiim u tahay
- Xawaaraha iyo miisaanka. Socodka shaqada luqaddu si dhakhso leh ayay u socon kartaa iyada oo aan la hurayn joogteynta.
- Helitaanka iyo gaarsiinta. Waxay balaadhisaa gelitaanka luqadaha iyo qaababka isgaarsiinta.
- Go'aamo cad. Kooxuhu waxay waqti badan ku qaadan karaan xukunka halka otomaatiggu uu qabanayo ku celcelinta.
Iska ilaali
- Xaqiiqooyinka dhalanteed waxay si deggan u geli karaan warbixinnada, taageerada socodka, ama natiijooyinka cilmi-baarista.
- Dareenka degdega ahi wuxuu abuuri karaa natiijooyin aan iswaafaqayn codsiyada la midka ah.
- Xogta qoraalka xasaasiga ah ayaa laga yaabaa in la kashifo haddii kontaroolada gelitaanka ay daciif yihiin.
Tallaabooyinka xiga
- Qeex qaabka wax soo saarka, codka, iyo heerarka tayada ka hor inta aan la baahin.
- Jawaabaha salka ku haya ilo lagu kalsoon yahay mar kasta oo saxnidu ay muhiim tahay.
- Hayso isbaarada dib u eegista bini aadamka ee wax soo saarka sare.
- Lasoco qaababka guuldarada oo dib u leyli dardargelinta ama socodka shaqada si joogto ah.
Sii wad Sahaminta
Warqad bilaash ah
Hel warbixinta maalinlaha ah ee AI
Saddex sheeko oo la xaqiijiyay oo AI ah maalin kasta oo shaqo ah, oo lagu qoray Ingiriis fudud. Bilaash had iyo jeer, aan xayaysiis lahayn.
Hal iimayl maalin kasta oo shaqo ah. Ka bax hal gujin. Marna ma iibinno ama ma wadaagno ciwaankaaga.
Is tijaabi
Take the Masked Language Modeling quiz
Fikrad degdeg ah oo ku saabsan jawaab kasta, iyo shahaado la wadaajin karo oo leh aqoonsi la xaqiijin karo markaad ka gudubto koorso.
Taageer waxbarashada AI bilaashka ah. AI Understanding waa 501(c)(3) aan faa'iido doon ahayn — xayaysiis la'aan, albaab lacag ah ma jiro, had iyo jeer. Samee deeq
Su'aalaha soo noqnoqda
What is Masked Language Modeling?
Masked language modeling (MLM) trains a bidirectional encoder to predict selected input tokens from surrounding context. BERT uses a token-corruption scheme; later methods alter the masking or prediction objective.
Waa maxay shaqada aasaasiga ah ee tababarka ee qaabaynta luqadda daboolan?
MLM waxa uu qariyaa jajab yar oo calaamado ah wuxuuna tababaraa qaabka si uu u soo kabsado iyaga oo isticmaalaya macnaha bidix iyo midig labadaba.
Qiyaastii boqolkiba meeqa calaamadood ayay BERT caadi ahaan maaskarada u qarisaa xilliga tababarka ka horreeya?
BERT waxay daboolaysaa qiyaastii 15% calaamadaha gelinta, dheelitirka u dhexeeya bixinta calaamad ku filan iyo ka tagista xaalad ku filan.
Muxuu MLM u bixiyaa qaabka fahamka laba jiho?
Beddelka beddelka wuxuu isticmaalaa is-fiirsi aan sabab lahayn, markaa calaamad kasta waxay arki kartaa dhammaan kuwa kale ee labada dhinacba.
Qorshaha waji-xidhka ee BERT, maxaa ku dhacaya ilaa 10% boosaska la doortay intii ay noqon lahaayeen [MASK]?
Si loo hagaajiyo adkaanta, 10% boosaska wejiyada daboolan ayaa lagu beddelaa calaamad random 10% ayaa laga tagayaa iyadoo aan isbeddelin.
Meelkee lagu xisaabiyaa khasaaraha MLM?
Luminta-entropy-ka-dhaafka ah waxaa lagu dabaqaa oo keliya meelaha waji-xidhan, marka la barbar dhigo saadaasha aqoonsigii asalka ahaa.
Sii wad waxbarashada
Tilmaamaha la xidhiidha
Tilmaamayaal badan ayaa loo doortay mawduucan