ELECTRA Pretraining
ELECTRA ita ce hanya mafi inganci don horar da ƙirar harshe ta hanyar koya musu su gano kalmomi na karya maimakon yin hasashen na ɓoye.
Dubawa
It matches BERT's quality using a fraction of the compute.
Zurfafa nutsewa
ELECTRA (Ingantacciyar Koyan Rubutun da ke Rarraba Maye gurbin Token Daidai), wanda Google da Stanford suka gabatar a cikin 2020, ya maye gurbin aikin ƙirar-harshen masked na BERT tare da 'masanin gano alamar.' Ƙaramar cibiyar sadarwa ta janareta tana musanya wasu kalmomi a cikin jumla don wasu madaidaitan hanyoyi, kuma babban abin ƙira (mai nuna bambanci) ya koyi yanke shawara, ga kowane alama ɗaya, ko na asali ne ko maye gurbinsa. Saboda samfurin yana horo akan dukkan alamu maimakon ~15% kawai wanda BERT ke rufewa, yana koyo da sauri. An ba da rahoton ELECTRA-Small ya zarce GPT mai girman kwatankwacin wanda aka horar tare da ƙarin lissafin 30x, da ELECTRA-Large RoBERTa da XLNet akan ma'aunin GLUE yayin amfani da kusan kashi ɗaya cikin huɗu na lissafin.
Fahimtar Fasaha
Tiransfoma biyu suna yin horo tare. Janareta yana yin ƙirar harshe mai rufe fuska kuma yana ba da shawarar maye gurbin alamu; mai wariya yana yin rabe-raben binary (ainihin vs. maye gurbinsu) akan kowane matsayi. Mahimmanci, ana ƙididdige asarar a kan dukkan alamu, ba kawai masu rufe fuska ba, suna ba da siginar koyo mai zurfi. Abubuwan haɗin da aka haɗa da hannun jari guda biyu, janareta yana ƙanƙanta (sau da yawa kwata zuwa rabin girman girman mai nuna bambanci), kuma bayan an riga an riga an yi watsi da janareta - kawai mai wariyar yana daidaitawa a ƙasa.
Dabarun Tasiri
Gudu da sikelin
Gudun aikin harshe na iya tafiya da sauri ba tare da sadaukar da daidaito ba.
Shiga ku isa
Yana faɗaɗa damar shiga cikin harsuna da salon sadarwa.
Shawarwari masu haske
Ƙungiyoyi za su iya ciyar da ƙarin lokaci akan hukunci yayin da aiki da kai ke sarrafa maimaitawa.
Makomar ELECTRA Pretraining
Ra'ayin gano alama-alamar maye gurbin ELECTRA ya rinjayi ingantattun incoders kamar DeBERTa-v3, wanda ya haɗa shi da kulawar da ba ta dace ba don sakamako na zamani. Kamar yadda ƙungiyoyi ke kula da tsadar horo da sawun carbon, makasudin horarwa na wariya waɗanda ke matse sigina daga kowace alama sun kasance masu ban sha'awa don gina ƙaƙƙarfan ƙaƙƙarfan ƙira. Yi tsammanin tsarin da za a ci gaba da sanar da ƙanana, samfura masu sauri don binciken kan na'ura, rarrabuwa, da kuma dawo da su inda manyan samfuran ƙira suka yi yawa.
Aiwatar da Gaskiyar Duniya
Ƙarfafa rarrabuwar rubutu da sauri da kuma nazarin ji a inda ake buƙatar ƙarami, ingantacciyar rikodi
Yin hidima azaman ƙashin baya don dacewar bincike da tsarin martabar daftarin aiki
Kyakkyawan daidaitawa ELECTRA-Ƙananan don na'ura ko ƙananan ayyuka na NLP tare da iyakanceccen ƙididdigewa.
Yin aiki azaman mai rikodin tushe mai ƙarfi don tantance sunan mahaɗan da alamomin amsa tambaya kamar SQuAD da GLUE
Hatsari & Tsare-tsare
Abubuwan da aka ruɗe suna iya shigar da rahotanni cikin nutsuwa, kwararar tallafi, ko abubuwan bincike.
Hankali na gaggawa na iya ƙirƙirar sakamako mara daidaituwa a cikin buƙatun iri ɗaya.
Za a iya fallasa bayanan rubutu mai ma'ana idan ikon samun dama yana da rauni.
Taswirar Hanya
Ƙayyade tsarin fitarwa, sautin, da ma'auni masu inganci kafin fitowa.
Amsa a ƙasa tare da amintattun tushe a duk lokacin da daidaito ya shafi mahimmanci.
Ajiye wurin binciken ɗan adam don abubuwan da ake samu masu girma.
Bibiyar tsarin gazawar kuma sake horar da tsokaci ko tafiyar aiki akai-akai.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the ELECTRA Pretraining quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
XLNet Permutation Modeling
Tambayoyin da ake yawan yi
What is ELECTRA Pretraining?
ELECTRA ita ce hanya mafi inganci don horar da ƙirar harshe ta hanyar koya musu su gano kalmomi na karya maimakon yin hasashen na ɓoye. Ya yi daidai da ingancin BERT ta amfani da juzu'i na lissafin.
Wane aikin horarwa ne ELECTRA ke amfani da shi maimakon ƙirar harshe mai rufe fuska?
ELECTRA tana horar da ƙirar don gano waɗanne alamu ne aka maye gurbinsu da janareta, maimakon tsinkaya kalmomin da aka rufe.
Me yasa ELECTRA ta fi BERT aiki da lissafi?
Mai wariya yana rarraba kowane alamar a matsayin gaske ko maye gurbinsa, don haka ƙirar tana koya daga 100% na matsayi maimakon ɓangaren da aka rufe kawai.
Wace rawa karamar cibiyar sadarwa ta janareta ke takawa a cikin ELECTRA?
Janareta yana yin ƙirar harshe da abin rufe fuska kuma yana ba da alamun karya na gaskiya, ƙirƙirar aikin ga mai nuna bambanci.
Menene zai faru da janareta bayan an gama pretraining?
Mai nuna wariya ne kawai aka adana kuma an daidaita shi don ayyuka na ƙasa; an jefar da janareta.
ELECTRA an samo asali ne daga masu bincike daga wane kungiyoyi?
An gabatar da ELECTRA a cikin 2020 ta masu bincike a Google da Jami'ar Stanford.