Kwerekana Ururimi
Ururimi rwerekana maskike rwigisha AI kuzuza amagambo yihishe nkana ukoresheje imiterere yuzuye, ibumoso n'iburyo.
Incamake
It's the training trick behind BERT and the reason models can deeply understand sentence meaning rather than just predict what comes next.
Kwibira cyane
Mu kwerekana imvugo yerekana ibimenyetso (MLM), ufata interuro, uhishe uhisha hafi 15% yikimenyetso cyayo hamwe nikimenyetso kidasanzwe [MASK], hanyuma uhugura icyitegererezo cyo gukeka umwimerere. Kuberako icyitegererezo kibona amagambo kumpande zombi za ubusa, yubaka ibyerekezo byombi kubyerekeranye. BERT, yatangijwe na Google muri 2018, yamenyekanye cyane. Ubwenge burambuye: bwimyanya yipfundikiriye, hafi 80% bahinduka [MASK], 10% bahinduranya ijambo ridahwitse, naho 10% basigaye badahindutse. Ibi birinda icyitegererezo kutigera dutegereza gusa ikimenyetso [MASK] mugihe cyo guhanura kandi kigatera imbaraga. Nyuma yo kwitegura, icyitegererezo cyateguwe neza kubikorwa nko gutondekanya, gusubiza ibibazo, no kumenyekana-kumenyekana.
Ubushishozi
MLM ikoresha kodegisi ya Transformer hamwe no kwiyitaho byombi, bityo buri kimenyetso cyitabira abandi icyarimwe. Igihombo kibarwa gusa kumyanya yipfutse ukoresheje cross-entropy irwanya indangamuntu nyayo. Kuberako kwitabwaho atari impamvu (nta masike izaza), guhagararirwa kuri buri jambo bihuza ibumoso niburyo muburyo bumwe. Ibyo byerekezo byombi nibyo rwose-moderi ikurikira ireka ubushobozi bwo kubyara.
Ingaruka z'Ingamba
Umuvuduko n'igipimo
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika.
Kugera no kugera
Yagura uburyo bwindimi nuburyo bwo gutumanaho.
Ibyemezo bisobanutse
Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo.
Ejo hazaza h'ururimi rwuzuye Masked
MLM isukuye igice cyafashwe na moderi yerekana decoder yerekana ibiganiro, ariko iracyiganje mugushira, kugarura, no gutondekanya aho gusobanukirwa gukubita ibisekuruza. Ibihinduka nka RoBERTa, ELECTRA yasimbuwe-token detection, na DeBERTa komeza usunike neza kandi neza. Tegereza kodegisi ya MLM kugirango igume hagati yo gushakisha, guhuza ibisobanuro, no nkibintu byoroheje imbere muri sisitemu nini yo kugarura-kwongerwa na sisitemu nyinshi aho gusobanukirwa byihuse, byimbitse kuruta inyandiko-yubusa.
Gushyira mu bikorwa Isi
Guha imbaraga Google Shakisha BERT ishingiye kubisobanuro byibibazo byo kuganira kugirango usubize impapuro zingirakamaro.
Gukora interuro yashizwemo gushakisha ibisobanuro hamwe na sisitemu yo kugarura inyandiko.
Kuringaniza neza BERT yo gusesengura amarangamutima kubicuruzwa cyangwa amatike yo gushyigikira.
Izina-ryamenyekanye kumenyekanisha abantu, amashyirahamwe, n'amatariki bivuye mumategeko cyangwa ubuvuzi.
Ingaruka & Kurinda
Ibintu bifatika bishobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi.
Kwihuta byihuse birashobora gukora ibisubizo bidahuye mubisabwa bisa.
Ibyanditswe byumvikana birashobora kugaragara niba kugenzura kugenzura ari ntege.
Igishushanyo mbonera
Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira.
Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro.
Komeza kugenzura abantu kugenzura ibisubizo byinshi.
Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Masked Language Modeling quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Kwerekana Ururimi
Ibibazo bikunze kubazwa
What is Masked Language Modeling?
Ururimi rwerekana maskike rwigisha AI kuzuza amagambo yihishe nkana ukoresheje imiterere yuzuye, ibumoso n'iburyo. Nuburyo bwamahugurwa inyuma ya BERT nimpamvu abanyamideli bashobora kumva cyane ibisobanuro byinteruro aho guhanura ibizakurikiraho.
Ni ubuhe butumwa bwibanze bwo guhugura mu kwerekana ururimi rwihishwa?
MLM ihisha agace k'ibimenyetso kandi itoza icyitegererezo cyo kugarura ukoresheje ibumoso n'iburyo.
Hafi ya ijanisha ryibimenyetso BERT isanzwe ihisha mugihe cyo kwitegura?
BERT yerekana hafi 15% yinjiza ibimenyetso, impirimbanyi hagati yo gutanga ibimenyetso bihagije no gusiga imiterere ihagije.
Kuki MLM itanga icyitegererezo cyicyerekezo cyombi?
Transformer encoder ikoresha idafite impamvu yo kwiyitaho, bityo buri kimenyetso gishobora kubona izindi zose kumpande zombi.
Muri gahunda ya mask ya BERT, bigenda bite nka 10% yimyanya yatoranijwe aho kuba [MASK]?
Kugirango utezimbere imbaraga, 10% yimyanya yipfundikirwa ihindurwamo ikimenyetso kidasanzwe kandi 10% isigaye idahindutse.
Ni iyihe myanya igihombo cya MLM kibarwa?
Igihombo cya cross-entropy gikoreshwa gusa kumyanya yapimwe, ugereranije ibyahanuwe nindangamuntu yumwimerere.