Ọzụzụ ELECTRA
ELECTRA bụ ụzọ dị mma iji zụọ ụdị asụsụ site n'ịkụziri ha ịhụ mkpụrụokwu adịgboroja kama ịkọ nke zoro ezo.
Nchịkọta
It matches BERT's quality using a fraction of the compute.
Ime miri emi
ELECTRA (ịmụ ihe nke ọma nke ọma nke na-edochi ngbanwe Token nke ọma), nke Google na Stanford webatara na 2020, jiri 'nchọpụta ihe nnochite anya dochie anya ọrụ BERT''s masked-asụsụ. Otu obere netwọk generator na-agbanwe okwu ụfọdụ n'ahịrịokwu maka ihe ndị ọzọ dị mma, na isi ihe nlereanya (onye ịkpa ókè) na-amụta ikpebi, maka otu akara ọ bụla, ma ọ bụ nke mbụ ma ọ bụ dochie ya. N'ihi na ihe nlereanya a na-azụ na token niile karịa naanị ~ 15% nke BERT na-ekpuchi, ọ na-amụta ngwa ngwa. A kọrọ na ELECTRA-Small ga-eme nke ọma karịa GPT nke zụrụ ya na 30x karịa, yana ELECTRA-Large RoBERTa na XLNet gbara ọsọ na GLUE benchmark mgbe ọ na-eji ihe dị ka otu ụzọ n'ụzọ anọ nke mgbakọ ahụ.
Nghọta nka nka
Transformers abụọ na-azụkọ ọnụ. Igwe ọkụ na-eme nhazi asụsụ masked ma na-atụpụta akara nnọchi anya; onye ịkpa ókè na-eme nhazi ọnụọgụ abụọ (ezigbo vs. dochie anya) karịa ọnọdụ ọ bụla. N'ụzọ dị oke mkpa, a na-agbakọ mfu ya na akara niile, ọ bụghị naanị nke kpuchiri ekpuchi, na-enye akara mmụta mmụta siri ike. Abụọ òkè token agbakwunyere, generator na-nọ na-dị obere (mgbe a nkeji iri na ise na ọkara nke discriminator size), na mgbe pretraining generator na-atụfu - naanị discriminator na-ọma-na-ege ntị downstream.
Mmetụta atụmatụ
Ọsọ na ọnụ ọgụgụ
Usoro ọrụ asụsụ nwere ike ịga ngwa ngwa n'achụghị nkwụsi ike.
Nweta na iru
Ọ na-agbasawanye ohere n'ofe asụsụ na ụdị nzikọrịta ozi.
Mkpebi doro anya
Ndị otu nwere ike itinyekwu oge na ikpe ebe akpaaka na-ejikwa nkwughachi.
Ọdịnihu nke ELECTRA Pretraining
Echiche nchọpụta ihe ngbanwe nke ELECTRA metụtakwara ndị ngbanwe dị mma dị ka DeBERTa-v3, nke jikọtara ya na nlebara anya enweghị atụ maka nsonaazụ ọgbara ọhụrụ. Dị ka òtù dị iche iche na-enwe mmasị karị maka ụgwọ ọzụzụ na akara ụkwụ carbon, ebumnobi izizi ịkpa oke nke na-amanye akara sitere na akara ọ bụla na-adọrọ mmasị maka iwulite ihe mkpuchi siri ike, kọmpat. Na-atụ anya ka a ga-esi gaa n'ihu na-agwa ụdị obere ngwa ngwa maka ịchọ ngwaọrụ, nhazi ọkwa, na iweghachite ebe nnukwu ụdị mmepụta na-emebiga ihe ókè.
Mmejuputa n'ezie n'ụwa
Na-akwado nhazi ederede ngwa ngwa yana nyocha mmetụta ebe achọrọ kọmpat, koodu ziri ezi
Na-eje ozi dị ka ọkpụkpụ azụ maka mkpa ọchụchọ na usoro nhazi akwụkwọ
ELECTRA-obere na-emezigharị nke ọma maka ngwaọrụ ma ọ bụ ọrụ NLP dị ala nwere obere mgbako.
Na-eme dị ka ihe ngbanwe ntọala siri ike maka njirimara aha-onye na akara azịza ajụjụ dịka SQuAD na GLUE
Ihe ize ndụ & okporo ụzọ nche
Eziokwu ndị e chepụtara echepụta nwere ike jiri nwayọ tinye akụkọ, nkwado nkwado, ma ọ bụ nsonaazụ nyocha.
Mmetụta ngwa ngwa nwere ike ịmepụta nsonaazụ na-ekwekọghị ekwekọ n'ofe arịrịọ ndị yiri ya.
Enwere ike ikpughe data ederede nwere mmetụta ma ọ bụrụ na njikwa ohere adịghị ike.
Map mmejuputa
Kọwaa usoro mmepụta, ụda, na ụkpụrụ ịdịmma tupu ibugharị.
Weghachite nzaghachi site na isi mmalite ntụkwasị obi mgbe ọ bụla izi ezi dị mkpa.
Debe ebe nleba anya mmadụ maka mpụta dị elu.
Sochie ụkpụrụ ọdịda ma na-azụghachi mkpali ma ọ bụ usoro ọrụ mgbe niile.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the ELECTRA Pretraining quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Nlereanya Permutation XLNet
Ajụjụ a na-ajụkarị
What is ELECTRA Pretraining?
ELECTRA bụ ụzọ dị mma iji zụọ ụdị asụsụ site n'ịkụziri ha ịhụ mkpụrụokwu adịgboroja kama ịkọ nke zoro ezo. Ọ dabara n'ịdịmma BERT site na iji ntakịrị nke compute.
Kedu ọrụ nkuzi tupu ELECTRA na-eji kama ịmegharị asụsụ kpuchiri ekpuchi?
ELECTRA na-azụ ihe nlere ahụ iji chọpụta akara ndị ejiri igwe na-emepụta ihe dochie anya, kama ibu amụma okwu kpuchie.
Kedu ihe kpatara ELECTRA ji gbakọọ-arụ ọrụ nke ọma karịa BERT?
Onye ịkpa oke na-ekewa akara ngosi ọ bụla dị ka ezigbo ma ọ bụ dochie ya, yabụ ihe nlereanya ahụ na-amụta site na 100% nke ọnọdụ kama ịbụ naanị subset nwere mkpuchi.
Kedu ọrụ obere netwọk generator na-arụ na ELECTRA?
Igwe ọkụ na-eme nhazi asụsụ kpuchiri ekpuchi ma na-enye akara adịgboroja adịgboroja, na-eke ọrụ maka ndị ịkpa oke.
Kedu ihe na-eme generator ka emechara ọzụzụ izizi?
Naanị onye ịkpa oke ka a na-edobe ma na-edozi ya maka ọrụ mgbada; a na-atụfu generator.
Ndị ọrụ nyocha sitere na kedu otu weputara ELECTRA?
Ndị nyocha na Google na Mahadum Stanford webatara ELECTRA na 2020.