Icyitegererezo cya XLNet
XLNet ihuza ibyerekezo byombi bya BERT hamwe na autoregressive yahanuye GPT mumahugurwa hejuru yo gutondekanya ijambo.
Incamake
This permutation trick lets it learn from all positions without ever masking tokens.
Kwibira cyane
XLNet, yatangijwe muri 2019 na Carnegie Mellon na Google Ubwonko, bwakozwe kugirango bukosore inenge muburyo bwa BERT. BERT yerekana ibimenyetso bya BERT kandi irabihanura, ariko ikimenyetso cyibihimbano [MASK] ntigishobora kugaragara mugihe cyo gutunganya neza, gukora gari ya moshi / ikizamini kidahuye, kandi BERT ikeka ko ibimenyetso byapfunditswe byigenga. XLNet ahubwo ikoresha 'permisation y'ururimi rwerekana imiterere': iragaragaza cyane ibyateganijwe byinjira-bishoboka hejuru yamagambo yose ashoboka yamagambo akurikirana. Muguhanura buri kimenyetso cyahawe igice cyabandi, icyitegererezo kibona neza ibyerekezo byombi mugihe gikomeje kuba moderi ikwiye idafite mask. Yubatswe kumugongo wa Transformer-XL kugirango yibuke intera ndende, XLNet yarushije BERT kumirimo igera kuri 20 harimo gusubiza ibibazo, gusesengura amarangamutima, no gutondekanya inyandiko.
Ubushishozi
XLNet ntabwo ihindura amagambo; iremera gahunda yo gutondeka ikoresheje masike yo kwitondera, amakuru yamakuru rero arabitswe. Kugirango ukore iki gikorwa, ikoresha 'ibice bibiri-byo-kwiyitaho': umurongo wibirimo urimo ibimenyetso byombi hamwe nibisobanuro byacyo, hamwe numurongo wibibazo uzi aho intego ihagaze ariko ntabwo ibiyirimo, bigafasha guhanura udatanze igisubizo. Guhindura-XL kwisubiramo hamwe na kodegisi ya posisiyo igereranya itanga kwibuka mubice birebire, kunoza imikorere yinyandiko ndende.
Ingaruka z'Ingamba
Umuvuduko n'igipimo
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika.
Kugera no kugera
Yagura uburyo bwindimi nuburyo bwo gutumanaho.
Ibyemezo bisobanutse
Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo.
Kazoza ka XLNet Uruhushya rwo Kwerekana
XLNet yari gihamya ikomeye yerekana ko intego za autoregressive zishobora gufata ibyerekezo byombi, bigahindura BERT-na GPT igabana. Mugihe umurima ahanini wahujwe hafi ya kodegisi zipfundikijwe cyangwa decodeur nini ya autoregressive, igitekerezo cyo kwemerera XLNet hamwe na Transformer-XL byagarutsweho byamenyesheje nyuma imirimo yo kwerekana imiterere ndende hamwe n'intego zihuriweho. Ubushishozi bwabwo bukomeza kuba ingirakamaro mugihe abashakashatsi bashakisha imyubakire ihuza imiterere ikomeye yerekana imiterere hamwe nibisekuru bidafite mask.
Gushyira mu bikorwa Isi
Kugera kubisubizo byo hejuru kubipimo bisubiza ibibazo nkibipimo nka SQuAD
Gukemura inyandiko-ndende imirimo nka RACE gusoma-gusobanukirwa ikizamini ukoresheje Transformer-XL yibuka
Imbaraga zinyandiko zikurikirana hamwe na sisitemu-yo kugarura amakuru
Gutezimbere amarangamutima gutondekanya no gutondekanya inyandiko hejuru ya BERT
Ingaruka & Kurinda
Ibintu bifatika bishobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi.
Kwihuta byihuse birashobora gukora ibisubizo bidahuye mubisabwa bisa.
Ibyanditswe byumvikana birashobora kugaragara niba kugenzura kugenzura ari ntege.
Igishushanyo mbonera
Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira.
Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro.
Komeza kugenzura abantu kugenzura ibisubizo byinshi.
Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the XLNet Permutation Modeling quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Icyitegererezo
Ibibazo bikunze kubazwa
What is XLNet Permutation Modeling?
XLNet ihuza ibyerekezo byombi bya BERT hamwe na autoregressive yahanuye GPT mumahugurwa hejuru yo gutondekanya ijambo. Amayeri yo kwemerera kureka kwigira kumyanya yose utarinze guhisha ibimenyetso.
Niyihe ntego yo kwitegura XLNet ikoresha?
XLNet yerekana ibyateganijwe byinjira-byemewe hejuru yimpushya zose zerekana ibimenyetso, byitwa imvugo yemewe.
Ni izihe ntege nke za BERT XLNet yagenewe gukemura?
Ikimenyetso cya BERT [MASK] ntigishobora kugaragara mugihe cyo gutunganya neza kandi gifata ibyahanuwe bipfutse nkibigenga; XLNet yirinda ibibazo byombi.
Ni ubuhe butumwa bwa XLNet bwo-kwiyitaho butandukanye?
Ibirimo bikubiyemo ibimenyetso hamwe nibisobanuro, mugihe ikibazo cyibibazo kimenya intego yintego ariko ntigikubiyemo, gishobora guhanura nta kumeneka.
Ese XLNet ihindura umubiri mumagambo?
XLNet yemerera guhanura (factorisation) ikoresheje mask yo kwitondera; imyanya yumwimerere imyanya ibitswe binyuze muri kodegisi.
Ni ubuhe bwubatsi bukora nkumugongo wa XLNet kumurongo muremure?
XLNet yubaka kuri Transformer-XL, yongeraho igice gisubiramo hamwe na kodegisi ihagaze kugirango ikore ibintu birebire.