ELMo Contextual Embeddings
ELMo (Embeddings kubva kuMutauro Models) yaive budiriro ye2018 iyo yakapa izwi rimwe nerimwe chimiro chakaumbwa nemutsara waro, saka 'bhangi' mu 'river bank' rinosiyana ne 'bhangi' mu 'savings bank.' Yakaratidza kuchinja kubva kune static izwi vectors kuenda kumamiriro-anoziva NLP.
Kudzika Kwakadzika
ELMo, yakaunzwa neAllen Institute yevaongorori veAI (Peters et al., 2018), inogadzira zvinomiririra mazwi nekumhanyisa mutsara kuburikidza neyakadzama bidirectional LSTM mutauro modhi yakadzidziswa pabhiriyoni-shoko corpus. Kusiyana neWord2Vec kana GloVe, iyo inopa imwe yakagadziriswa vector pashoko, ELMo inokokorodza vhekita nyowani yezvese zvinoitika zvichibva pane zvakatenderedza mamiriro. Zvine hutsinye, ELMo inosanganisa ese emukati LSTM akaturikidzana kuburikidza neakadzidzwa, basa-rakananga huremu pane kushandisa chete yepamusoro layer. Matunhu epasi anowanzo kutora syntax (chikamu-che-kutaura, chimiro) ukuwo maturu epamusoro achitora semantics uye pfungwa yezwi. Kuwedzera ELMo kumamodheru aripo kwakaburitsa mibairo mikuru pamabasa matanhatu ebhenji, kusanganisira kupindura mibvunzo, kuongorora manzwiro, uye kuzivikanwa kwesangano.
Technical Insight
ELMo inorongedza maLSTM maviri: muenzaniso wemutauro wekumberi uchifanotaura izwi rinotevera uye yekumashure ichifanotaura izwi rapfuura, rimwe nerimwe pamusoro pemabhii-level CNN maiputs (saka inobata mazwi asingaonekwe). Kuti uite basa rezasi, ELMo inodonhedza ratidziro ichishandisa zviremu zvakapfava-zvakajairwa pamwe ne scalar, zvese zvakadzidzwa panguva yekumisikidza. Izvi zvinoreva kuti basa rega rega rinogona kusarudza kuti yakawanda sei syntactic kupesana nesemantic chiratidzo chainoda kubva kuchando pretrained biLM.
Strategic Impact
Kumhanya uye chiyero
Mutauro workflows inogona kufamba nekukurumidza pasina kupira kuenderana.
Svika uye svika
Inopamhidzira kupinda mumitauro yese nemataera ekutaurirana.
Sarudzo dzakajeka
Zvikwata zvinogona kupedza nguva yakawanda pakutonga uku otomatiki ichibata kudzokorora.
Ramangwana reELMo Contextual Embeddings
Pfungwa huru yeELMo, inomiririra yemutauro-modhi yekudzidzira, yakave hwaro, asi iyo yakadzokororwa LSTM dhizaini yakakurumidza kuvharwa neTransformer-based modhi seBERT mukupera kwa2018, iyo inoverenga mitsara yakazara mukuwirirana uye kukwira zvakanyanya. Nhasi ELMo inonyanya kukosha kwenhoroondo uye yedzidzo, kunyange zvazvo maitiro-CNN mabatiro ekuisa uye kuyera-kurema mazano achiri kupesvedzera basa rakasarudzika mumitauro yakaderera uye morphologically yakapfuma.
Real-World Implementation
Kuvandudza masisitimu ekuzivikanwa kwenzvimbo inofanirwa kutaura kana 'Washington' ichireva munhu, nyika, kana guta zvichibva pamashoko akatenderedza.
Kuwedzera ongororo yemanzwiro nekutora kuti 'kurwara' kunoreva kuti 'ndinonzwa kurwara' asi zvakanaka mune slang 'iyo inorwara'
Kuvandudza masisitimu ekupindura mibvunzo pane SQUAD bhenji nekudyisa mamiriro-anonzwa-sensitive token vectors muverengi.
Kusiyanisa manzwisisiro emazwi mushanduro yemuchina zvekuti mazwi ekuti 'chirimwa' anoturikira mamiriro akapihwa nemazvo
Njodzi & Guardrails
Chokwadi chehuroyi chinogona kupinda chinyararire mishumo, kuyerera kwetsigiro, kana tsvakiridzo.
Kunzwa nekukasira kunogona kugadzira mhedzisiro isingaenderane pane zvikumbiro zvakafanana.
Sensitive text data inogona kuburitswa kana zvidhiraivho zvisina kusimba.
Implementation Roadmap
Tsanangura chimiro chekubuda, toni, uye mhando zviyero usati waburitsa.
Mhinduro dzepasi neakavimbika masosi pese pazvine basa.
Chengetedza ongororo yekuongorora yemunhu kune yakakwira-stake zvinobuda.
Tevera maitiro ekutadza uye dzidzisazve kukurudzira kana mafambiro ebasa nguva nenguva.
Ramba Uchiongorora
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the ELMo Contextual Embeddings quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gaidhi rinotevera
Shoko Embeddings
Mibvunzo inowanzo bvunzwa
What is ELMo Contextual Embeddings?
ELMo (Embeddings kubva kuMutauro Models) yaive budiriro ye2018 iyo yakapa izwi rimwe nerimwe chimiro chakaumbwa nemutsara waro, saka 'bhangi' mu 'river bank' rinosiyana ne 'bhangi' mu 'savings bank.' Yakaratidza kuchinja kubva kune static izwi vectors kuenda kumamiriro-anoziva NLP.
Ndeupi mutsauko wakakosha pakati peELMo nekutanga embeddings seWord2Vec?
ELMo inogadzira embeddings yemamiriro: iyo vector yezwi inoshanduka zvichienderana nemutsara wakatenderedza, kusiyana neWord2Vec's single fixed vector pazwi.
Ndeipi neural architecture inoshandiswa neELMo kuverenga zvinyorwa?
ELMo yakavakirwa pane yakadzika bidirectional LSTM yakadzidziswa semuenzaniso wemutauro, kugadzirisa kutevedzana kuchidzokororwa.
ELMo inosanganisa sei zvikamu zvayo zvemukati zvebasa rezasi?
ELMo inodzidza basa-chaiyo softmax-yakajairwa uremu kusanganisa ese biLM akaturikidzana, ichirega basa rega rega risimbise syntax kana semantics sezvinodiwa.
Chii chinoshandiswa neELMo sezvikamu zvayo zvekupinza kubata mazwi asingaonekwe?
ELMo inovaka mazwi ekupinda kubva kuhunhu-level CNN, saka inogona kumiririra mazwi asina kumboona panguva yekudzidziswa.
Zvinenge kuti ELMo yakaunzwa rinhi uye nani?
ELMo yakaburitswa muna 2018 naPeter et al. paAllen Institute yeAI (AI2).