Samfuran Mara Kyau da Ƙimar Ƙimar Haɓaka Amo
Samfuran mara kyau da Ƙimar Ƙirar Haɗin Kai (NCE) dabaru ne waɗanda ke barin ƙila su koyi manyan ƙamus ba tare da ƙididdige cikakken farashi mai tsada ba.
Dubawa
Instead of scoring every possible output, they teach the model to tell real (positive) examples from a handful of fake (negative) ones.
Zurfafa nutsewa
Lokacin da ƙamus yana da dubban ɗaruruwan kalmomi, madaidaicin softmax na yau da kullun dole ne ya daidaita kowace kalma don kowane matakin horo - a hankali sosai. Hayaniyar Contrastive Kiyasin yana warware matsalar azaman rarrabuwa na binary: da aka ba da manufa da kuma wasu samfuran 'hayoyi' da aka zana daga sanannen rarraba, koyi bambanta samfurin gaskiya daga amo, wanda a fakaice ya dawo da yuwuwar da ake so ba tare da daidaitawa ba. Samfuran mara kyau, wanda aka shahara ta samfurin skip-gram na word2vec, ɗan uwa ne mai sauƙaƙa: ga kowane gaskiya (kalmar, mahallin) ya haɗa samfuran k negatives kuma yana horar da ƙirar don sanya babban maki ga ainihin biyu da ƙananan maki ga karya, ta amfani da manufar sigmoid. Dukansu suna juya matsala mai arha mai tsada zuwa masu arha da yawa, suna yin babban horo na haɗawa da aiki. Zaɓin rarraba amo (sau da yawa unigram ya tashi zuwa ikon 3/4) yana rinjayar inganci sosai.
Fahimtar Fasaha
NCE tana ƙididdige ƙima ta hanyar rarrabuwar bayanai da hayaniya, kuma yayin da adadin samfuran amo ya ƙaru yana iya kusantar matsakaicin yuwuwar tare da ingantaccen ingantaccen softmax. Samfur mara kyau yana sauke sharuɗɗan daidaitawa na NCE gaba ɗaya, yana inganta log σ(tabbatacciyar maƙiya) + Σ log σ(-maki mara kyau). Wannan ya sa ya yi sauri amma ba madaidaicin ƙididdige yawan ƙididdigewa ba - an sa shi don koyon ingantattun abubuwan sakawa maimakon ƙima. Samfuran abubuwan da ba su dace ba daga sassauƙan rarraba juzu'i (mita^0.75) yana daidaita ma'auni na gama-gari da na yau da kullun.
Dabarun Tasiri
Kudin da kasafin kuɗi
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Shawarwari masu haske
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Kula da inganci
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Makomar Samfurin Mara Kyau da Ƙimar Ƙimar Harutu
Babban ra'ayin - koyo ta hanyar banbanta tabbatacce game da abubuwan da aka zayyana - a yanzu suna ba da kulawa ta zamani da ilmantarwa na wakilci a cikin hangen nesa, harshe, da shawarwari. Aiki na gaba yana mai da hankali kan hakar ma'adinai mara kyau (zaɓan mummunan bayanai maimakon na bazuwar), ɓata ra'ayi don abubuwan da ba su dace ba, da zazzage mummunan rahusa ta hanyar manyan bankunan ƙwaƙwalwar ajiya ko samfurin cikin-tsari. Kamar yadda samfura ke girma, ingantattun maƙasudin samfuri sun kasance masu mahimmanci a duk inda wuraren fitarwa ko saitunan ɗan takara suke da yawa, kamar maidowa da manyan masu ba da shawara.
Aiwatar da Gaskiyar Duniya
word2vec skip-gram tare da ƙaddamar da kalmomin koyo mara kyau daga biliyoyin alamu ba tare da cikakken softmax ba.
Samfuran harshe a tarihi suna amfani da NCE don horar da ƙamus na dubban ɗaruruwan kalmomi cikin inganci.
Nasiha da tsarin dawo da samfurin abubuwan 'marasa kyau' mai amfani bai yi mu'amala da su ba don horar da ƙirar hasumiya biyu.
Haruffa da zane-zane na ilimi (misali, lalata kai ko wutsiya sau uku) ta amfani da samfurori mara kyau don koyon alaƙar mahalli.
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Negative Sampling and Noise Contrastive Estimation quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Kan layi da Ma'adinai mara kyau
Tambayoyin da ake yawan yi
What is Negative Sampling and Noise Contrastive Estimation?
Samfuran mara kyau da Ƙimar Ƙirar Haɗin Kai (NCE) dabaru ne waɗanda ke barin ƙila su koyi manyan ƙamus ba tare da ƙididdige cikakken farashi mai tsada ba. Maimakon nuna maki kowane abin da zai yiwu, suna koya wa ƙirar don faɗi misalai na gaske (tabbatacce) daga ɗimbin na karya (mara kyau).
Wace matsala ce samfurin rashin kyau da NCE ke magance?
Dukansu hanyoyin suna guje wa daidaitawa a kan babban ƙamus ta hanyar sake fasalin horo azaman wariya mai rahusa.
Ta yaya Ƙimar Ƙarfafa Harutu ke tsara aikin koyo?
NCE tana horar da ƙirar don bambanta samfuran gaske daga samfuran da aka zana daga sanannen rarraba amo.
Wanne samfuri ne ya shahara da ƙima mara kyau don koyon haɗa kalmomi?
An gabatar da samfur mara kyau kuma ya shahara ta bambancin skip-gram na word2vec.
Ta yaya samfur mara kyau ya bambanta da cikakken NCE?
Samfur mara kyau yana sauƙaƙa NCE ta hanyar cire al'ada, ciniki mai yiwuwa don saurin gudu da ingantaccen kayan haɗi.
Me yasa ake yawan yin samfura daga rabe-raben rabe-rabe zuwa ikon 3/4?
Ƙarfin 0.75 yana daidaita rarraba mitar don haka kalmomin gama gari ba sa mamaye kuma kalmomin da ba kasafai suke bayyana ba har yanzu.