Model Harshen Marufi
Tsarin harshe da aka rufe da rufe yana koyar da AI don cika kalmomin ɓoye da gangan ta amfani da cikakkun mahallin kewaye, hagu da dama.
Dubawa
It's the training trick behind BERT and the reason models can deeply understand sentence meaning rather than just predict what comes next.
Zurfafa nutsewa
A cikin ƙirar harshe da aka rufe (MLM), kuna ɗaukar jumla, ɓoye kusan kashi 15% na alamunta ba tare da wata alama ta [MASK] ba, kuma ku horar da ƙirar don tantance ainihin asali. Domin samfurin yana ganin kalmomi a ɓangarorin biyu na kowane fanko, yana gina fahimtar mahallin bidirectional. BERT, wanda Google ya gabatar a cikin 2018, ya yada wannan. Daki-daki mai wayo: na wuraren rufe fuska, kusan 80% sun zama [MASK], 10% ana musanya su don kalmar bazuwar, kuma 10% an bar su ba canzawa. Wannan yana hana ƙirar daga taɓa tsammanin alamar [MASK] a lokacin tsinkaya kuma yana tilasta ƙarfi. Bayan wannan horon na farko, samfurin yana da kyau a daidaita shi don ayyuka kamar rarrabuwa, amsa tambayoyi, da tantance sunan mahaɗan.
Fahimtar Fasaha
MLM yana amfani da mai canza canji tare da kulawar kai na bidirectional, don haka kowane alama yana halartar duk sauran lokaci guda. Ana ƙididdige asarar a kan wuraren da aka rufe kawai ta amfani da giciye-entropy a kan ainihin ID ɗin alama. Saboda hankali ba shi da dalili (babu abin rufe fuska na gaba), wakilcin kowace kalma ta haɗa mahallin hagu da dama zuwa cikin ɗaki ɗaya. Wannan bidirectionality shine ainihin abin da ƙila na gaba ya daina don ikon samarwa.
Dabarun Tasiri
Gudu da sikelin
Gudun aikin harshe na iya tafiya da sauri ba tare da sadaukar da daidaito ba.
Shiga ku isa
Yana faɗaɗa damar shiga cikin harsuna da salon sadarwa.
Shawarwari masu haske
Ƙungiyoyi za su iya ciyar da ƙarin lokaci akan hukunci yayin da aiki da kai ke sarrafa maimaitawa.
Makomar Samar da Harshen Marufi
MLM mai tsafta an rufe shi ta hanyar ƙirar ƙirar ƙirar ƙira don chatbots, amma ya kasance mai rinjaye don haɗawa, dawo da, da rarrabuwa inda fahimta ke bugun tsara. Bambance-bambancen kamar RoBERTa, gano alamar maye gurbin ELECTRA, da DeBERTa suna ci gaba da tura daidaito da inganci. Yi tsammanin encoders irin na MLM su kasance tsakiyar bincike, kamancen mazhabobi, kuma a matsayin sassauƙa masu nauyi a cikin manyan abubuwan da aka haɓaka da haɓakawa da tsarin multimodal inda sauri, zurfin fahimta ya fi dacewa da rubutu na kyauta.
Aiwatar da Gaskiyar Duniya
Ƙaddamar da Google Bincike na tushen BERT na fahimtar tambayoyin tattaunawa don dawo da ƙarin shafuka masu dacewa.
Ƙirƙirar abubuwan haɗin jumla don bincike na ma'ana da tsarin dawo da daftari.
Kyakkyawan daidaitawa BERT don nazarin ra'ayi akan bitar samfur ko tikitin tallafi.
Ƙaddamar da sunan sunan wanda ke fitar da mutane, ƙungiyoyi, da kwanan wata daga rubutun doka ko likita.
Hatsari & Tsare-tsare
Abubuwan da aka ruɗe suna iya shigar da rahotanni cikin nutsuwa, kwararar tallafi, ko abubuwan bincike.
Hankali na gaggawa na iya ƙirƙirar sakamako mara daidaituwa a cikin buƙatun iri ɗaya.
Za a iya fallasa bayanan rubutu mai ma'ana idan ikon samun dama yana da rauni.
Taswirar Hanya
Ƙayyade tsarin fitarwa, sautin, da ma'auni masu inganci kafin fitowa.
Amsa a ƙasa tare da amintattun tushe a duk lokacin da daidaito ya shafi mahimmanci.
Ajiye wurin binciken ɗan adam don abubuwan da ake samu masu girma.
Bibiyar tsarin gazawar kuma sake horar da tsokaci ko tafiyar aiki akai-akai.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Masked Language Modeling quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Samfuran Harshe
Tambayoyin da ake yawan yi
What is Masked Language Modeling?
Tsarin harshe da aka rufe da rufe yana koyar da AI don cika kalmomin ɓoye da gangan ta amfani da cikakkun mahallin kewaye, hagu da dama. Dabarar horarwa ce a bayan BERT kuma dalilin ƙira na iya zurfin fahimtar ma'anar jumla maimakon kawai hasashen abin da ke gaba.
Menene ainihin aikin horarwa a cikin ƙirar harshe da aka rufe?
MLM yana ɓoye ɗan guntun alamomi kuma yana horar da ƙirar don dawo da su ta amfani da mahallin hagu da dama.
Kusan kashi nawa ne na alamomin BERT ke rufe fuska yayin horo?
BERT yana rufe kusan kashi 15% na alamun shigarwa, ma'auni tsakanin bada isasshiyar sigina da barin isasshiyar mahallin.
Me yasa MLM ke ba da samfurin fahimtar juna biyu?
Mai canza canji yana amfani da hankalin kai wanda ba dalili ba, don haka kowane alama zai iya ganin duk sauran bangarorin biyu.
A cikin tsarin rufe fuska na BERT, menene zai faru da kusan kashi 10% na mukamai da aka zaɓa maimakon zama [MASK]?
Don inganta ƙarfi, 10% na wuraren rufe fuska ana musanya su don alamar bazuwar kuma an bar 10% ba canzawa.
A kan waɗanne matsayi aka ƙididdige asarar MLM?
Ana amfani da asarar giciye-entropy zuwa wuraren da aka rufe kawai, kwatanta tsinkaya zuwa ID na asali na alama.