Overview
Negative sampling uye Noise Contrastive Estimation (NCE) matipi anoita kuti mamodheru adzidze pamusoro pemazwi makuru pasina komputa inodhura yakazara softmax. Panzvimbo yekupa zvibodzwa zvese zvinogoneka, vanodzidzisa modhi kutaura chaiyo (yakanaka) mienzaniso kubva kune mashoma emanyepo (asina kunaka).
Negative Sampling uye Noise Contrastive Estimation chivakwa chehunyanzvi chinobata mhando yemhando, mutengo wezvivakwa, latency, uye kuvimbika pachiyero.
Deep Dive
Kana izwi riine mazana ezviuru zvemashoko, yakajairwa softmax inofanirwa kudzikamisa pamusoro peshoko rega rega padanho rega rega rekudzidzisa - kunonoka zvakanyanya. Noise Contrastive Estimation inogadziridza dambudziko sechikamu chebhinari: yakapihwa chinangwa uye mashoma 'ruzha' samples anotorwa kubva kune inozivikanwa kugovera, dzidza kusiyanisa yechokwadi sampuli kubva kune ruzha, iyo inodzoreredza zvinogoneka zvinodikanwa zvingangoitika pasina kujeka kujaira. Negative sampling, yakakurumbira nezwi2vec's skip-gram modhi, ihama yakarerutswa: kune yega yega yechokwadi (izwi, mamiriro) pairi iyo sampuli k zvisizvo uye inodzidzisa modhi kupa yakakwira zvibodzwa kune chaiyo vaviri uye yakaderera zvibodzwa kune iyo fake, uchishandisa sigmoid chinangwa. Ose ari maviri anoshandura dambudziko rinodhura remakirasi akawanda kuita akawanda akachipa mabhinari, zvichiita kuti kudzidziswa kwakakura-kudzika kushande. Sarudzo yekugovera ruzha (kazhinji unigram yakasimudzwa kune iyo 3/4 simba) inokanganisa zvakanyanya kunaka.
Technical Insight
NCE inofungidzira modhi nekuisa dhata maringe neruzha, uye sezvo huwandu hwemasampunzi eruzha hunokura hunogona kuenzanisa mukana wepamusoro-kana ine yakajairwa yakajairwa softmax. Negative sampling inodonhedza mazwi eNCE's normalization zvachose, optimize log σ(positive mamakisi) + Σ log σ(−negative mamakisi). Izvo zvinoita kuti ikurumidze asi isisiri inopindirana density estimator - yakarongedzerwa yekudzidza zvakanaka embeddings pane zvakayerwa zvingangoitika. Sampling negatives kubva kune yakatsetseka unigram kugovera (frequency^0.75) inoyera mazwi akajairika uye asingawanzo.
Kubata Negative Sampling uye Noise Contrastive Estimation
Negative sampling uye Noise Contrastive Estimation (NCE) matipi anoita kuti mamodheru adzidze pamusoro pemazwi makuru pasina komputa inodhura yakazara softmax. Panzvimbo yekupa zvibodzwa zvese zvinogoneka, vanodzidzisa modhi kutaura chaiyo (yakanaka) mienzaniso kubva kune mashoma emanyepo (asina kunaka). Negative Sampling uye Noise Contrastive Estimation chivakwa chehunyanzvi chinobata mhando yemhando, mutengo wezvivakwa, latency, uye kuvimbika pachiyero. Kuvaka kunzwisisa kwakadzama, bata Negative Sampling uye Noise Contrastive Estimation semuenzaniso wekushandisa, kwete chinhu chimwe chete: tsanangura zvaunoda mhedzisiro, kujekesa fungidziro, uye patsanura zvinogona kuitwa nehurongwa hwakavimbika kubva kune zvichiri kuda kutonga kwenyanzvi.
Mukuita, zvikwata zvakasimba zvinoshandisa Negative Sampling uye Noise Contrastive Estimation inogonesa zvivakwa, data, uye sarudzo dzezvivakwa zvinopesana nekuvimbika uye mutengo. Ivo vanonyora zvakajeka maitiro ebudiriro, bvunzo vachipokana ne data rechokwadi uye mafambiro ebasa, uye iterate zvichibva pane zvakacherechedzwa maitiro ekutadza kwete kuhwina-nguva imwe chete yebhenji. Apa ndipo apo kunzwisisa kwe theoretical kunoshanduka kuve kugona kwakasimba pane chigadzirwa, mutemo, uye mashandiro.
Zvisarudzo zvezvivakwa zvinotyaira kuita uye mutengo wekushandisa kwemakore. Panguva imwecheteyo, Kukwirisa imwe bhenji kunogona kuvanza yakafara system kushaya simba. Nzira yakatsiga ndeyekubatanidza kukurumidza kuyedza nekutonga: mhanyisa vatyairi vendege, tora humbowo, buritsa matanda esarudzo, uye urambe uchivandudza chengetedzo semaitiro emuenzaniso, zvinotarisirwa nemushandisi, uye zvinodikanwa zvekutonga.
Strategic Impact
Zvisarudzo zvezvivakwa zvinotyaira kuita uye mutengo wekushandisa kwemakore.
Zvisarudzo zvezvivakwa zvinotyaira kuita uye mutengo wekushandisa kwemakore. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Dzidzo yehunyanzvi inobatsira zvikwata kusarudza murwi wakakodzera, kwete iwo mutsva chete.
Dzidzo yehunyanzvi inobatsira zvikwata kusarudza murwi wakakodzera, kwete iwo mutsva chete. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Sarudzo dzeinjiniya dziri nani dzinoderedza zviitiko zvekuvimbika mukugadzira.
Sarudzo dzeinjiniya dziri nani dzinoderedza zviitiko zvekuvimbika mukugadzira. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Real-World Implementation
word2vec skip-gram ine negative sampling yekudzidza mazwi embeddings kubva kumabhiriyoni ematokeni pasina yakazara softmax.
Mamodheru emitauro kare akashandisa NCE kudzidzisa pamusoro pemashoko emazana ezviuru zvemashoko zvinobudirira.
Kurudziro nekutora masisitimu ekuenzanisira 'negative' zvinhu zvisina mushandisi kudzidzisa mamodheru ekumisikidza shongwe mbiri.
Girafu uye ruzivo-girafu embeddings (semuenzaniso, kukanganisa musoro kana muswe katatu) uchishandisa sampuli dzisina kunaka kudzidza hukama hwemasangano.
Maitiro Ekuita
Negative Sampling uye Noise Contrastive Estimation mukuita
word2vec skip-gram ine negative sampling yekudzidza mazwi embeddings kubva kumabhiriyoni ematokeni pasina yakazara softmax.
word2vec skip-gram ine yakashata sampling yekudzidza mazwi ekumisikidza kubva kumabhiriyoni ematokeni asina akazara softmax Matimu anowanzo kuwana mhedzisiro iri nani kana achinge atsanangura hunhu hwepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi emupendero, uye kuteedzera zvese zvakawanikwa zvechigadzirwa uye mutengo wekukanganisa nekufamba kwenguva.
Negative Sampling uye Noise Contrastive Estimation mukuita
Mamodheru emitauro kare akashandisa NCE kudzidzisa pamusoro pemashoko emazana ezviuru zvemashoko zvinobudirira.
Mitauro modhi yaimboshandisa NCE kudzidzisa pamusoro pemashoko emazana ezviuru emashoko nemazvo Matimu anowanzo kuwana mhedzisiro iri nani kana achinge atsanangura hunhu hwepamberi, chengetedza nzira yekukwira kwevanhu yemakesi ekumucheto, uye kuteedzera zvese zvakawanikwa zvechigadzirwa nemitengo yekukanganisa nekufamba kwenguva.
Negative Sampling uye Noise Contrastive Estimation mukuita
Kurudziro nekutora masisitimu ekuenzanisira 'negative' zvinhu zvisina mushandisi kudzidzisa mamodheru ekumisikidza shongwe mbiri.
Kurudziro uye kudzoreredza masisitimu ekuenzanisa 'negative' zvinhu izvo mushandisi asina kudyidzana nazvo kudzidzisa maviri-shongwe yekumisikidza modhi Matimu anowanzo kuwana mibairo iri nani kana achinge atsanangura emhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi emupendero, uye kuteedzera zvese zvakawanikwa zvechigadzirwa uye kukanganisa mutengo nekufamba kwenguva.
Negative Sampling uye Noise Contrastive Estimation mukuita
Girafu uye ruzivo-girafu embeddings (semuenzaniso, kukanganisa musoro kana muswe katatu) uchishandisa sampuli dzisina kunaka kudzidza hukama hwemasangano.
Girafu uye ruzivo-girafu kumisikidzwa (semuenzaniso, kukanganisa musoro kana muswe) uchishandisa sampuli dzisina kunaka kudzidza hukama hwemasangano Matimu anowanzo kuwana mibairo iri nani kana achinge atsanangura hunhu hwepamberi, chengetedza nzira yekukwira kwevanhu yemakesi emupendero, uye kuteedzera zvese zvakawanikwa zvechigadzirwa nemitengo yekukanganisa nekufamba kwenguva.
Njodzi & Guardrails
Kugadzirisa imwe bhenji kunogona kuvanza yakafara system kushaya simba.
Infrastructure uye mari yekugadzirisa inowanzotarisirwa pasi.
Chengetedzo uye kucherechedzwa mapundu anogona kukura sezvo masisitimu anowedzera kuoma.
Implementation Roadmap
Tsanangura latency, mhando, uye mutengo zvinangwa usati waitwa.
Tsanangura latency, mhando, uye mutengo zvinangwa usati waitwa. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Benchmark pasi pechokwadi mutoro uye data mamiriro.
Benchmark pasi pechokwadi mutoro uye data mamiriro. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Chishandiso chekutarisa zvikanganiso, kudonha, uye mushandisi maitiro.
Chishandiso chekutarisa zvikanganiso, kudonha, uye mushandisi maitiro. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Gadzirira nzira dzekudzosera kumashure uye dzezviitiko usati wawedzera.
Gadzirira nzira dzekudzosera kumashure uye dzezviitiko usati wawedzera. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.