InoteveraGaidhi rinotevera
Online uye Zvakaoma Negative Mining
Tekinoroji
Nhungamiro yehunyanzvi
Negative Sampling uye Noise Contrastive Estimation
Negative sampling uye Noise Contrastive Estimation (NCE) matipi anoita kuti mamodheru adzidze pamusoro pemazwi makuru pasina komputa inodhura yakazara softmax.
Teerera kugwaro iriAI-yakarondedzerwa · ~2 min
Pa peji ino2 min verenga
Pfupiso
Instead of scoring every possible output, they teach the model to tell real (positive) examples from a handful of fake (negative) ones.
Kudzika Kwakadzika
Kana izwi riine mazana ezviuru zvemashoko, yakajairwa softmax inofanirwa kudzikamisa pamusoro peshoko rega rega padanho rega rega rekudzidzisa - kunonoka zvakanyanya. Noise Contrastive Estimation inogadziridza dambudziko sechikamu chebhinari: yakapihwa chinangwa uye mashoma 'ruzha' samples anotorwa kubva kune inozivikanwa kugovera, dzidza kusiyanisa yechokwadi sampuli kubva kune ruzha, iyo inodzoreredza zvinogoneka zvinodikanwa zvingangoitika pasina kujeka kujaira. Negative sampling, yakakurumbira nezwi2vec's skip-gram modhi, ihama yakarerutswa: kune yega yega yechokwadi (izwi, mamiriro) pairi iyo sampuli k zvisizvo uye inodzidzisa modhi kupa yakakwira zvibodzwa kune chaiyo vaviri uye yakaderera zvibodzwa kune iyo fake, uchishandisa sigmoid chinangwa. Ose ari maviri anoshandura dambudziko rinodhura remakirasi akawanda kuita akawanda akachipa mabhinari, zvichiita kuti kudzidziswa kwakakura-kudzika kushande. Sarudzo yekugovera ruzha (kazhinji unigram yakasimudzwa kune iyo 3/4 simba) inokanganisa zvakanyanya kunaka.
Kuitwa muNyika yeChokwadi
word2vec skip-gram ine negative sampling yekudzidza mazwi embeddings kubva kumabhiriyoni ematokeni pasina yakazara softmax.
Mamodheru emitauro kare akashandisa NCE kudzidzisa pamusoro pemashoko emazana ezviuru zvemashoko zvinobudirira.
Kurudziro nekutora masisitimu ekuenzanisira 'negative' zvinhu izvo mushandisi asina kupindirana nazvo kudzidzisa mamodheru ekumisikidza shongwe mbiri.
Girafu uye ruzivo-girafu embeddings (semuenzaniso, kukanganisa musoro kana muswe katatu) uchishandisa sampuli dzisina kunaka kudzidza hukama hwemasangano.
Ramangwana reNegative Sampling uye Noise Contrastive Estimation
Pfungwa yepakati - dzidza nekusiyanisa mapositi kubva kune ega ega - ikozvino inotsigisa yemazuva ano yekuzvitarisira uye inosiyanisa inomiririra kudzidza pane yekuona, mutauro, uye kurudziro. Basa remangwana rinotarisa pane zvakaoma-zvisina kunaka migodhi (kusarudza zvinodzidzisa zvisina kunaka panzvimbo pezvisina kurongeka), kurerekera kune zvisirizvo, uye kuyera zvisina kunaka zvakachipa kuburikidza nehombe ndangariro mabhangi kana mu-batch sampling. Sezvo mamodheru achikura, zvibodzwa zvesampled zvinoshanda zvinoramba zvichikosha chero nzvimbo dzekubuda kana seti dzevamiriri dzakakura, sekudzosa uye vakurudziri vakakura.
Kuzvishandisa
Nei zvichikosha
- Mutengo uye bhajeti. Zvisarudzo zvezvivakwa zvinotyaira kuita uye mutengo wekushandisa kwemakore.
- Sarudzo dzakajeka. Dzidzo yehunyanzvi inobatsira zvikwata kusarudza murwi wakakodzera, kwete iwo mutsva chete.
- Kudzora kwemhando yepamusoro. Sarudzo dzeinjiniya dziri nani dzinoderedza zviitiko zvekuvimbika mukugadzira.
Chenjera
- Kugadzirisa imwe bhenji kunogona kuvanza yakafara system kushaya simba.
- Infrastructure uye mari yekugadzirisa inowanzotarisirwa pasi.
- Chengetedzo uye kucherechedzwa mapundu anogona kukura sezvo masisitimu anowedzera kuoma.
Matanho anotevera
- Tsanangura latency, mhando, uye mutengo zvinangwa usati waitwa.
- Benchmark pasi pechokwadi mutoro uye data mamiriro.
- Chishandiso chekutarisa zvikanganiso, kudonha, uye mushandisi maitiro.
- Gadzirira nzira dzekudzosera kumashure uye dzezviitiko usati wawedzera.
Ramba Uchiongorora
Tsamba yemahara
Tora iyo yemazuva ese AI pfupiso
Nhatu dzakasimbiswa dzeAI nyaya mangwanani ega ega pakati pevhiki, dzakanyorwa neChirungu chakajeka. Yemahara zvachose, hapana ads.
Imwe email zuva rega rega revhiki. Bvisa kunyoresa kamwe chete. Hatimbotengese kana kugovera kero yako.
Zviedze iwe pachako
Take the Negative Sampling and Noise Contrastive Estimation quiz
Mhinduro yepakarepo pamhinduro yega yega, uye chitupa chinogovaniswa chine ID inobatika kana wapedza kosi.
Tsigira dzidzo yemahara yeAI. AI Understanding ndeye 501(c)(3) isingabatsiri - hapana ads, hapana paywall, zvachose. Ita chipo
Mibvunzo inowanzo bvunzwa
What is Negative Sampling and Noise Contrastive Estimation?
Negative sampling uye Noise Contrastive Estimation (NCE) matipi anoita kuti mamodheru adzidze pamusoro pemazwi makuru pasina komputa inodhura yakazara softmax. Panzvimbo yekupa zvibodzwa zvese zvinogoneka, vanodzidzisa modhi kutaura chaiyo (yakanaka) mienzaniso kubva kune mashoma emanyepo (asina kunaka).
Idambudziko ripi rinoitwa zvisina kunaka sampling uye NCE zvinonyanya kugadzirisa?
Maitiro ese ari maviri anodzivirira kujairana pamusoro pemazwi makuru nekugadzirisa dzidziso seyakachipa yebhinari rusarura.
Ko Noise Contrastive Estimation inogadziridza sei basa rekudzidza?
NCE inodzidzisa modhi kusiyanisa masampuli chaiwo kubva kumasampuli akatorwa kubva kune inozivikanwa ruzha kugovera.
Ndeupi modhi yakakurumbira sampling isina kunaka yekudzidza kumisikidzwa kwemazwi?
Negative sampling yakaunzwa uye yakakurumbira neskip-gram musiyano wezwi2vec.
Ko sampling yakaipa inosiyana sei neNCE yakazara?
Negative sampling inorerutsa NCE nekubvisa normalization, kutengeserana probabilistic kurongeka kwekumhanya uye zvakanaka embeddings.
Sei zvisina kunaka zvichiwanzo kutorwa kubva pakugovera unigram kusimudzwa kune iyo 3/4 simba?
Iyo 0.75 exponent inotsvedza kugoverwa kwefrequency kuti mazwi akajairika haatongi uye mazwi asingawanzo achiri kuoneka.
Ramba uchidzidza
Mazano anoenderana
Mamwe madhairekitori akasarudzirwa nyaya iyi