Ukumodela Ulimi Olufihliwe
Ukumodela kolimi olufihliwe kufundisa i-AI ukuthi igcwalise amagama afihliwe ngamabomu isebenzisa umongo ogcwele ozungezile, kokubili kwesokunxele nakwesokudla.
Uhlolojikelele
It's the training trick behind BERT and the reason models can deeply understand sentence meaning rather than just predict what comes next.
I-Deep Dive
Kumodeli yolimi olufihlekile (MLM), uthatha umusho, ufihle ngokungahleliwe cishe u-15% wamathokheni awo ngophawu olukhethekile lwe-[MASK], bese uqeqesha imodeli ukuqagela okwangempela. Ngenxa yokuthi imodeli ibona amagama nhlangothi zombili zokungenalutho ngakunye, yakha ukuqonda okukabili komongo. I-BERT, yethulwa ngu-Google ngo-2018, ikwenze kwaduma lokhu. Imininingwane ehlakaniphile: ezindaweni ezimboziwe, cishe u-80% uba [MASK], u-10% ushintshaniswa ngegama elingahleliwe, futhi u-10% ushiywa ungashintshiwe. Lokhu kuvimbela imodeli ukuthi ingalindeli kuphela ithokheni ye-[MASK] ngesikhathi sokubikezela futhi kuphoqa ukuqina. Ngemva kwalokhu kuqeqeshwa kwangaphambili, imodeli ilungiselelwa kahle imisebenzi efana nokuhlukanisa, ukuphendula imibuzo, nokuqashelwa kwebhizinisi.
I-Technical Insight
I-MLM isebenzisa isishumeki seTransformer esinokuzinaka okukabili, ngakho yonke ithokheni ibheka wonke amanye ngesikhathi esisodwa. Ukulahlekelwa kubalwa kuphela ezindaweni ezifihliwe kusetshenziswa i-cross-entropy ngokumelene nama-ID wamathokheni eqiniso. Ngenxa yokuthi ukunaka akusona isizathu (akukho ukufihla ubuso esikhathini esizayo), ukumelwa kwegama ngalinye kuhlanganisa umongo kwesokunxele nowesokudla kube ivekhtha eyodwa eminyene. Lokho kuqondanisa kabili yilokho kanye amamodeli amathokheni alandelayo anikela ngakho ukuze akwazi ukukhiqiza.
I-Strategic Impact
Isivinini nesikali
Ukugeleza komsebenzi wolimi kungahamba ngokushesha ngaphandle kokudela ukuvumelana.
Finyelela futhi ufinyelele
Yandisa ukufinyelela kuzo zonke izilimi nezitayela zokuxhumana.
Izinqumo ezicacile
Amaqembu angachitha isikhathi esiningi ekwahluleleni kuyilapho i-automation isingatha impinda.
Ikusasa Lokumodela Ulimi Olufihliwe
I-MLM ehlanzekile ngokwengxenye isithwe amamodeli edikhoda akhiqizayo wama-chatbots, kodwa isalokhu ilawula ukushumeka, ukubuyisa, nokuhlelwa lapho ukuqonda kudlula ukukhiqiza. Okuhlukile okufana ne-RoBERTa, ukutholwa kwethokheni esikhundleni se-ELECTRA, kanye ne-DeBERTa ilokhu igcizelela ukunemba nokusebenza kahle. Lindela izishumeki zesitayela se-MLM ukuthi zihlale zimaphakathi kusesho, ukufana kwe-semantic, futhi njengezingxenye ezingasindi ngaphakathi kwezinhlelo ezinkulu zokubuyisa-augmented kanye ne-multimodal lapho ukuqonda okusheshayo, okujulile kubaluleke ngaphezu kombhalo wefomu lamahhala.
Ukuqaliswa Komhlaba Wangempela
Inika amandla Google Ukusesha okusekwe ku-BERT ukuqonda kwemibuzo yengxoxo ukubuyisela amakhasi ahlobene kakhulu.
Ikhiqiza ukushumeka kwemisho kokusesha kwe-semantic kanye nezinhlelo zokubuyisa imibhalo.
I-BERT yokulungisa kahle ukuhlaziya imizwa ekubuyekezweni komkhiqizo noma amathikithi osekelo.
Ukuqashelwa okuqanjwe kwebhizinisi okukhipha abantu, izinhlangano, nezinsuku embhalweni wezomthetho noma wezokwelapha.
Izingozi & Guardrails
Amaqiniso akhonjiwe angafaka ngokuthula imibiko, ukugeleza kosekelo, noma imiphumela yocwaningo.
Ukuzwela okusheshayo kungadala imiphumela engahambisani kuzo zonke izicelo ezifanayo.
Idatha yombhalo ebucayi ingase idalulwe uma izilawuli zokufinyelela zibuthakathaka.
Ukuqalisa Umhlahlandlela
Chaza ifomethi yokuphumayo, ithoni, namazinga wekhwalithi ngaphambi kokukhishwa.
Izimpendulo eziyisisekelo ngemithombo ethembekile noma nini lapho ukunemba kubalulekile.
Gcina indawo yokuhlola isibuyekezo somuntu ukuze uthole imiphumela ephezulu.
Landela amaphethini okuhluleka futhi uqeqeshe kabusha imiyalo noma ukuhamba komsebenzi njalo.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Masked Language Modeling quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Umhlahlandlela olandelayo
Ukumodela Ulimi
Imibuzo evame ukubuzwa
What is Masked Language Modeling?
Ukumodela kolimi olufihliwe kufundisa i-AI ukuthi igcwalise amagama afihliwe ngamabomu isebenzisa umongo ogcwele ozungezile, kokubili kwesokunxele nakwesokudla. Iqhinga lokuqeqesha ngemuva kwe-BERT futhi amamodeli esizathu angaqonda ngokujulile incazelo yomusho kunokumane abikezele ukuthi yini ezayo ngokulandelayo.
Imuphi umsebenzi owumongo wokuqeqesha ekumodeleni kolimi olufihliwe?
I-MLM ifihla ingxenye yamathokheni futhi iqeqeshe imodeli ukuthi iwabuyisele kusetshenziswa kokubili umongo kwesokunxele nakwesokudla.
Cishe yimaphi amaphesenti amathokheni i-BERT ajwayele ukuwafihla ngesikhathi sokuqeqeshwa?
I-BERT imaski cishe u-15% wamathokheni okokufaka, ibhalansi phakathi kokunikeza isignali eyanele nokushiya umongo owanele.
Kungani i-MLM inikeza ukuqonda okubili okuyimodeli?
Isishumeki seTransformer sisebenzisa ukuzinaka okungesona isizathu, ukuze yonke ithokheni ikwazi ukubona zonke ezinye izinhlangothi zombili.
Kuhlelo lokufihla ubuso lwe-BERT, kwenzekani cishe ku-10% wezikhundla ezikhethiwe esikhundleni sokuba i-[MASK]?
Ukuze kuthuthukiswe ukuqina, u-10% wezikhundla ezifihliwe zishintshwa ukuze uthole ithokheni engahleliwe futhi u-10% ushiywe ungashintshiwe.
Yiziphi izikhundla lapho ukulahlekelwa kwe-MLM kubalwa khona?
Ukulahleka kwe-Cross-entropy kusetshenziswa kuphela ezindaweni ezisikiwe, kuqhathaniswa nezibikezelo kuma-ID wamathokheni okuqala.