I-XLNet Permutation Modeling
I-XLNet ihlanganisa umongo wokuqondiswa kabili we-BERT nokubikezela okuzenzakalelayo kwe-GPT ngokuqeqesha phezu kokuhleleka kwamagama okungahleliwe.
Uhlolojikelele
This permutation trick lets it learn from all positions without ever masking tokens.
I-Deep Dive
I-XLNet, yethulwe ngo-2019 nguCarnegie Mellon kanye ne-Google Brain, yakhelwe ukulungisa iphutha ekuqeqesheni kwangaphambili kwesitayela se-BERT. I-BERT imaski amathokheni futhi iyawabikezela, kodwa uphawu [MASK] lokwenziwa aluveli ngesikhathi sokushunwa kahle, okudala ukungafani kwesitimela/ukuhlola, futhi i-BERT ithatha ukuthi amathokheni ambozwe azimele. I-XLNet esikhundleni salokho isebenzisa 'i-permutation language modelling': ikhulisa amathuba okungena okulindelekile kukho konke ukuhleleka okungenzeka kwamagama ngokulandelana. Ngokubikezela ithokheni ngayinye enikezwe isethi engaphansi engahleliwe yamanye, imodeli ibona ngokuphumelelayo umongo we-bidirectional kuyilapho ihlezi iyimodeli efanele ye-autoregressive ngaphandle kokufihla ubuso. Yakhelwe phezu komgogodla we-Transformer-XL yenkumbulo yebanga elide, i-XLNet isebenze kahle ngokudlula i-BERT emisebenzini engaba ngu-20 ehlanganisa ukuphendula imibuzo, ukuhlaziya imizwa, kanye nezinga lemibhalo.
I-Technical Insight
I-XLNet ayishovi amagama ngokomzimba; ivumela i-oda le-factorization ngokusebenzisa imaski yokunaka, ngakho-ke ulwazi lwendawo luyagcinwa. Ukwenza lokhu kusebenze, isebenzisa 'ukuzinaka kokusakaza-mbili': ukusakaza kokuqukethwe okufaka ikhodi kokubili ithokheni nomongo wayo, kanye nokusakaza kombuzo owazi indawo yalowo oqondiwe kodwa hhayi okuqukethwe kwakho, okuvumela ukubikezela ngaphandle kokuvuza impendulo. Ukuphindaphinda kwe-Transformer-XL kanye nombhalo wekhodi wendawo ohlobene kuyinikeza inkumbulo kuwo wonke amasegimenti amade, ukuthuthukisa ukuphathwa kwamadokhumenti amade.
I-Strategic Impact
Isivinini nesikali
Ukugeleza komsebenzi wolimi kungahamba ngokushesha ngaphandle kokudela ukuvumelana.
Finyelela futhi ufinyelele
Yandisa ukufinyelela kuzo zonke izilimi nezitayela zokuxhumana.
Izinqumo ezicacile
Amaqembu angachitha isikhathi esiningi ekwahluleleni kuyilapho i-automation isingatha impinda.
Ikusasa Le-XLNet Permutation Modelling
I-XLNet yayiwubufakazi obunamandla bokuthi izinhloso ezizenzakalelayo zingathwebula umongo we-bidirectional, fiphaze ukuhlukana kwe-BERT-versus-GPT. Ngenkathi inkambu ihlanganiswe kakhulu phakathi kwama-encoder afihliweyo noma ama-decoder amakhulu azenzakalelayo, umbono wokuvumela we-XLNet kanye nokuphindaphinda kwe-Transformer-XL okwaziswa kamuva ngomsebenzi wokumodela womongo omude kanye nezinjongo ezihlanganisiwe zokuziqeqesha. Ukuqonda kwayo kuhlala kusebenza njengoba abacwaningi befuna izakhiwo ezihlanganisa imodeli eqinile yomongo nesizukulwane esisebenza kahle, esingenamaski.
Ukuqaliswa Komhlaba Wangempela
Ukuzuza imiphumela ephezulu kumabhentshimakhi wokuphendula imibuzo njenge-SQuAD
Ukuphatha imisebenzi yamadokhumenti amade njengokuhlola ukuqonda ukufunda kwe-RACE ngememori ye-Transformer-XL
Izilinganiso zamadokhumenti anamandla kanye nezinhlelo zokuthola ulwazi
Ukuthuthukisa ukuhlukaniswa kwemizwelo kanye nokuhlelwa kombhalo ngaphezu kwesisekelo se-BERT
Izingozi & Guardrails
Amaqiniso akhonjiwe angafaka ngokuthula imibiko, ukugeleza kosekelo, noma imiphumela yocwaningo.
Ukuzwela okusheshayo kungadala imiphumela engahambisani kuzo zonke izicelo ezifanayo.
Idatha yombhalo ebucayi ingase idalulwe uma izilawuli zokufinyelela zibuthakathaka.
Ukuqalisa Umhlahlandlela
Chaza ifomethi yokuphumayo, ithoni, namazinga wekhwalithi ngaphambi kokukhishwa.
Izimpendulo eziyisisekelo ngemithombo ethembekile noma nini lapho ukunemba kubalulekile.
Gcina indawo yokuhlola isibuyekezo somuntu ukuze uthole imiphumela ephezulu.
Landela amaphethini okuhluleka futhi uqeqeshe kabusha imiyalo noma ukuhamba komsebenzi njalo.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the XLNet Permutation Modeling quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Umhlahlandlela olandelayo
Ukumodela Kwesihloko
Imibuzo evame ukubuzwa
What is XLNet Permutation Modeling?
I-XLNet ihlanganisa umongo wokuqondiswa kabili we-BERT nokubikezela okuzenzakalelayo kwe-GPT ngokuqeqesha phezu kokuhleleka kwamagama okungahleliwe. Leli qhinga lemvume liyivumela ukuthi ifunde kuzo zonke izikhundla ngaphandle kokufihla amathokheni.
Iyiphi inhloso yokuqeqeshwa kwangaphambili esetshenziswa yi-XLNet?
I-XLNet ikhulisa amathuba okungena ngemvume okulindelekile phezu kwazo zonke izimvume ze-oda lethokheni le-factorization, elibizwa ngokuthi i-permutation language modelling.
Yibuphi ubuthakathaka be-BERT i-XLNet eyakhelwe ngokukhethekile ukubhekana nayo?
Uphawu lokwenziwa lwe-BERT [MASK] aluveli ngesikhathi sokulungiswa kahle futhi luphatha izibikezelo ezifihlekile njengezizimele; I-XLNet igwema zombili izinkinga.
Ukuzinaka okubili kwe-XLNet kwehlukanisa ngani?
Ukusakaza kokuqukethwe kubhala ithokheni nomongo, kuyilapho ukusakazwa kombuzo kwazi indawo yalowo okuqondiwe kodwa hhayi okuqukethwe kwakho, okunikeza amandla ukubikezela ngaphandle kokuvuza.
Ingabe i-XLNet ishova amagama emshweni?
I-XLNet ivumela ukubikezela (i-factorization) oda ngokusebenzisa imaski yokunaka; izindawo zamathokheni zangempela zigcinwa ngombhalo wekhodi wendawo.
Iyiphi i-architecture esebenza njengomgogodla we-XLNet kumongo webanga elide?
I-XLNet yakhela ku-Transformer-XL, eyengeza ukuphindaphinda kwesegimenti kanye nombhalo wekhodi ohlobene wokuphatha ukulandelana okude.