I-ELMo Contextual Embeddings Guide

I-Deep Dive

I-ELMo, eyethulwe yi-Allen Institute yabacwaningi be-AI (uPeters et al., 2018), ikhiqiza izethulo zamagama ngokusebenzisa umusho ngemodeli yolimi ye-LSTM ejulile yokuqondisa kabili eqeqeshwe kukhophasi yamagama ayibhiliyoni. Ngokungafani ne-Word2Vec noma i-GloVe, enikeza i-vector eyodwa engaguquki ngegama ngalinye, i-ELMo ibala i-vector entsha kukho konke ukwenzeka ngokusekelwe kumongo ozungezile. Okubalulekile, i-ELMo ihlanganisa zonke izendlalelo ze-LSTM zangaphakathi ngokusebenzisa izisindo ezifundiwe, eziqondene nomsebenzi othile kunokusebenzisa isendlalelo esiphezulu kuphela. Izendlalelo ezingezansi zivame ukuthatha i-syntax (ingxenye yenkulumo, isakhiwo) kuyilapho izendlalelo eziphakeme zithwebula i-semantics nomuzwa wamagama. Ukwengeza i-ELMo kumamodeli akhona kukhiqize izinzuzo ezinkulu kuyo yonke imisebenzi eyisithupha, okuhlanganisa ukuphendula imibuzo, ukuhlaziya imizwa, nokuqashelwa kwebhizinisi okuqanjwe igama.

I-Technical Insight

I-ELMo inqwabelanisa ama-LSTM amabili: imodeli yolimi oluya phambili ebikezela igama elilandelayo nengemuva elibikezela igama langaphambilini, ngalinye lingaphezu kokokufaka kwe-CNN kwezinga lohlamvu (ngakho iphatha amagama angabonakali). Ngomsebenzi owenzela ezansi nomfula, i-ELMo igoqa izethulo zesendlalelo isebenzisa izisindo ze-softmax-normalized kanye nesikala, konke okufundiwe phakathi nokushuna kahle. Lokhu kusho ukuthi umsebenzi ngamunye unganquma ukuthi ingakanani isignali ye-syntactic nesemantic oyifunayo ku-biLM eqandisiwe eqeqeshwe kusengaphambili.

I-Strategic Impact

Isivinini nesikali

Ukugeleza komsebenzi wolimi kungahamba ngokushesha ngaphandle kokudela ukuvumelana.

Finyelela futhi ufinyelele

Yandisa ukufinyelela kuzo zonke izilimi nezitayela zokuxhumana.

Izinqumo ezicacile

Amaqembu angachitha isikhathi esiningi ekwahluleleni kuyilapho i-automation isingatha impinda.

Ikusasa Lokushumeka Kokuqukethwe Kwe-ELMo

Umbono owumongo we-ELMo, izethulo zezingqikithi ezivela ekuqeqeshweni kwemodeli yolimi, zaba yisisekelo, kodwa ukwakheka kwayo okuphindaphindiwe kwe-LSTM kwasithwa ngokushesha amamodeli asuselwa ku-Transformer afana ne-BERT ngasekupheleni kuka-2018, efunda imisho yonke ngokuhambisana futhi yakha kangcono kakhulu. Namuhla i-ELMo ibaluleke kakhulu ngokomlando nezemfundo, nakuba ukuphatha okokufaka komlingiswa we-CNN kanye nemibono yokulinganisa ungqimba kusathonya umsebenzi wokushumeka okhethekile ngezilimi ezingenazinsiza eziphansi kanye nezilimi ezicebile ngokwesimo.

Ukuqaliswa Komhlaba Wangempela

Ukuthuthukisa amasistimu okuqashelwa kwebhizinisi okumele asho ukuthi i-'Washington' ibhekisela kumuntu, isifunda, noma idolobha ngokusekelwe emagameni azungezile.

Ukuthuthukisa ukuhlaziya imizwa ngokuthwebula ukuthi 'ogulayo' kusho ukuthi 'ngiyagula' kusho ukuthi 'ngiyagula' kodwa kuphoqelekile ngesiqubulo esithi 'lokho kuyagula'

Ukuthuthukisa amasistimu wokuphendula imibuzo kubhentshimakhi ye-SQUAD ngokuphakela ama-vector amathokheni azwela umongo kumfundi

Ukwehlukanisa imizwa yamagama ekuhumusheni komshini amagama amaningi afana 'nesitshalo' ahumusha kahle umongo onikeziwe

Izingozi & Guardrails

Amaqiniso akhonjiwe angafaka ngokuthula imibiko, ukugeleza kosekelo, noma imiphumela yocwaningo.

Ukuzwela okusheshayo kungadala imiphumela engahambisani kuzo zonke izicelo ezifanayo.

Idatha yombhalo ebucayi ingase idalulwe uma izilawuli zokufinyelela zibuthakathaka.

Ukuqalisa Umhlahlandlela

Chaza ifomethi yokuphumayo, ithoni, namazinga wekhwalithi ngaphambi kokukhishwa.

Izimpendulo eziyisisekelo ngemithombo ethembekile noma nini lapho ukunemba kubalulekile.

Gcina indawo yokuhlola isibuyekezo somuntu ukuze uthole imiphumela ephezulu.

Landela amaphethini okuhluleka futhi uqeqeshe kabusha imiyalo noma ukuhamba komsebenzi njalo.

Qhubeka Uhlole

ChatGPT ne-LLMs

Bona ukuthi amamodeli olimi esimanje akhiqiza kanjani futhi acabange.

Funda Umhlahlandlela

Izisekelo ze-NLP

Funda okuyisisekelo kokucubungula ulimi ngemuva kwalawa mathuluzi.

Funda Umhlahlandlela

Free newsletter

Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.

One email each weekday. Unsubscribe in one click. We never sell or share your address.

Test yourself

Take the ELMo Contextual Embeddings quiz

Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.

Qala imibuzo

Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation

Umhlahlandlela olandelayo

Ukushumeka kwamagama

Funda umhlahlandlela olandelayo Phequlula yonke imibuzo

Imibuzo evame ukubuzwa

What is ELMo Contextual Embeddings?

I-ELMo (Ukushumeka Okusuka Kumamodeli Olimi) kwaba impumelelo yango-2018 enikeza igama ngalinye ukumelwa okulotshwe umusho walo, ngakho 'ibhange' 'ebhange lomfula' lihlukile 'kubhange' elithi 'ibhange lokonga.' Kuphawule ukugudluka ukusuka kumavekhtha amagama amile ukuya ku-NLP eqaphela umongo.

Uyini umehluko obalulekile phakathi kwe-ELMo nokushumeka kwangaphambilini njenge-Word2Vec?

I-ELMo ikhiqiza ukushumeka kokuqukethwe: i-vector yegama iyashintsha ngokusekelwe emushweni ozungezile, ngokungafani nevekhtha eyodwa egxilile ye-Word2Vec ngegama ngalinye.

Iyiphi i-neural architecture esetshenziswa i-ELMo ukufunda umbhalo?

I-ELMo yakhelwe phezu kwe-LSTM ye-bidirectional ejulile eqeqeshwe njengemodeli yolimi, icubungula ukulandelana ngokuphindaphindiwe.

I-ELMo ihlanganisa kanjani izendlalelo zayo zangaphakathi ngomsebenzi ongezansi?

I-ELMo ifunda izisindo ze-softmax-normalized eziqondene nomsebenzi othile ukuhlanganisa zonke izendlalelo ze-biLM, ivumele umsebenzi ngamunye ugcizelele i-syntax noma i-semantics njengoba kudingeka.

I-ELMo isebenzisa ini njengamayunithi ayo okufaka ukuphatha amagama angabonakali?

I-ELMo yakha okokufaka kwamagama kusuka ku-CNN yezinga lohlamvu, ngakho ingamela amagama engazange iwabone ngesikhathi sokuqeqeshwa.

Cishe i-ELMo yethulwa nini futhi ubani?

I-ELMo yashicilelwa ngo-2018 nguPeters et al. e-Allen Institute for AI (AI2).