Nlezigharị Asụsụ Masked
Nlegharị anya asụsụ kpuchiri ekpuchi na-akuziri AI iji mejupụta okwu ezoro ezo na-eji ihe gbara ya gburugburu, ma aka ekpe ma aka nri.
Nchịkọta
It's the training trick behind BERT and the reason models can deeply understand sentence meaning rather than just predict what comes next.
Ime miri emi
N'ime ihe nlegharị anya asụsụ masked (MLM), ị na-ewere ahịrịokwu, jiri akara [MASK] pụrụ iche zoo ihe dị ka 15% nke akara ya, wee zụọ ihe nlereanya ahụ ka ọ mara ndị mbụ. N'ihi na ihe nlereanya ahụ na-ahụ okwu n'akụkụ abụọ nke oghere ọ bụla, ọ na-ewulite nghọta n'ụzọ abụọ nke ọnọdụ. BERT, nke Google webatara na 2018, mere ka nke a gbasaa. Nkọwa dị nkọ: nke ọnọdụ mkpuchi, ihe dịka 80% na-aghọ [MASK], 10% gbanwere maka okwu enweghị usoro, na 10% ka agbanweghị. Nke a na-egbochi ihe nlereanya ahụ ịtụ anya naanị akara [MASK] n'oge amụma ma na-amanye ike. Mgbe ọzụzụ izizi a gasịrị, a na-edobe ihe nlereanya a nke ọma maka ọrụ ndị dị ka nhazi ọkwa, ịza ajụjụ na nnabata aha.
Nghọta nka nka
MLM na-eji ihe ntụgharị ntụgharị nwere nlebara anya onwe onye bidirectional, yabụ akara ngosi ọ bụla na-aga ndị ọzọ niile n'otu oge. A na-agbakọ mfu ahụ naanị n'ọnọdụ mkpuchi mkpuchi site na iji cross-entropy megide ezigbo ID akara. N'ihi na nlebara anya abụghị ihe kpatara ya (enweghị ihe mkpuchi n'ọdịnihu), nnochite anya okwu ọ bụla na-ejikọta aka ekpe na aka nri n'ime otu vector siri ike. Na bidirectionality bụ kpọmkwem ihe na-esote-token ụdị na-ahapụ maka ike imepụta.
Mmetụta atụmatụ
Ọsọ na ọnụ ọgụgụ
Usoro ọrụ asụsụ nwere ike ịga ngwa ngwa n'achụghị nkwụsi ike.
Nweta na iru
Ọ na-agbasawanye ohere n'ofe asụsụ na ụdị nzikọrịta ozi.
Mkpebi doro anya
Ndị otu nwere ike itinyekwu oge na ikpe ebe akpaaka na-ejikwa nkwughachi.
Ọdịnihu nke Mwepụta Asụsụ Masked
MLM dị ọcha ejirila ụdị ihe ndozi ihe maka nkata kpuchiri ya, mana ọ ka na-achị maka ntinye, iweghachi na nhazi ọkwa ebe nghọta na-akụ ọgbọ. Ndị dị iche iche dị ka RoBERTa, nchọpụta akara edochi nke ELECTRA, na DeBERTa na-aga n'ihu na-akwalite izi ezi na arụmọrụ. Na-atụ anya ngbanwe ụdị MLM ka ha nọrọ n'etiti ịchọ, myirịta ọmụmụ, yana dị ka ihe ndị dị fechaa n'ime nnukwu iweghachite-agbakwunyere yana ọtụtụ sistemụ ebe ngwa ngwa, nghọta miri emi dị mkpa karịa ederede ụdị efu.
Mmejuputa n'ezie n'ụwa
Na-akwado Google Nghọta dabere na BERT nke ọchụchọ maka ajụjụ mkparịta ụka iji weghachi ibe ndị ọzọ dabara adaba.
Na-emepụta ntinye ahịrịokwu maka usoro nyocha na usoro iweghachi akwụkwọ.
Ndozi BERT dị mma maka nyocha mmetụta na nyocha ngwaahịa ma ọ bụ tiketi nkwado.
Nkwenye aha-aha nke na-ewepụta ndị mmadụ, otu, na ụbọchị site na ederede iwu ma ọ bụ ọgwụ.
Ihe ize ndụ & okporo ụzọ nche
Eziokwu ndị e chepụtara echepụta nwere ike jiri nwayọ tinye akụkọ, nkwado nkwado, ma ọ bụ nsonaazụ nyocha.
Mmetụta ngwa ngwa nwere ike ịmepụta nsonaazụ na-ekwekọghị ekwekọ n'ofe arịrịọ ndị yiri ya.
Enwere ike ikpughe data ederede nwere mmetụta ma ọ bụrụ na njikwa ohere adịghị ike.
Map mmejuputa
Kọwaa usoro mmepụta, ụda, na ụkpụrụ ịdịmma tupu ibugharị.
Weghachite nzaghachi site na isi mmalite ntụkwasị obi mgbe ọ bụla izi ezi dị mkpa.
Debe ebe nleba anya mmadụ maka mpụta dị elu.
Sochie ụkpụrụ ọdịda ma na-azụghachi mkpali ma ọ bụ usoro ọrụ mgbe niile.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Masked Language Modeling quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Nlereanya Asụsụ
Ajụjụ a na-ajụkarị
What is Masked Language Modeling?
Nlegharị anya asụsụ kpuchiri ekpuchi na-akuziri AI iji mejupụta okwu ezoro ezo na-eji ihe gbara ya gburugburu, ma aka ekpe ma aka nri. Ọ bụ aghụghọ ọzụzụ dị n'azụ BERT na ihe kpatara ụdị nwere ike ịghọta nke ọma ahịrịokwu pụtara kama ịkọ ihe na-esote.
Kedu ihe bụ isi ọrụ nkuzi na nhazi asụsụ ekpuchiri?
MLM na-ezobe ntakịrị ihe ngosi wee zụọ ihe nlereanya iji nwetaghachi ha site na iji ọnọdụ aka ekpe na aka nri.
Ihe dị ka pasent ole nke token BERT na-ekpuchikarị n'oge ọzụzụ?
BERT na-ekpuchi ihe dịka 15% nke akara ntinye, nguzozi n'etiti inye akara zuru oke na ịhapụ ọnọdụ zuru oke.
Kedu ihe kpatara MLM ji enye nghọta ihe nlere anya bidirectional?
Ihe ntụgharị ntụgharị na-eji nlebara anya onwe onye na-abụghị ihe kpatara ya, yabụ akara ngosi ọ bụla nwere ike ịhụ ndị ọzọ niile n'akụkụ abụọ.
N'ime atụmatụ mkpuchi BERT, kedu ihe na-eme ihe dịka 10% nke ọnọdụ ahọpụtara kama ịbụ [MASK]?
Iji kwalite nkwụsi ike, a na-agbanwe 10% nke ọnọdụ mkpuchi maka akara enweghị usoro yana 10% agaghị agbanwe agbanwe.
N'ọnọdụ ndị dị aṅaa ka a na-agbakọ mfu MLM?
A na-etinye mfu cross-entropy naanị na ọnọdụ mkpuchi, na-atụnyere amụma na NJ akara izizi.