Échantillonnage négatif ak xayma buy wuutale bruit
Échantillonnage négatif ak estimation contraste de bruit (NCE) ay pexe lañu yuy may model yi ñu jàng lu bari ci vocabulaire te duñu xayma softmax bu mat te seer lool.
Résumé
Instead of scoring every possible output, they teach the model to tell real (positive) examples from a handful of fake (negative) ones.
Plongeur bu xóot
Sudee vocabulaire amna téemeeri junni baat, softmax bu baax dafa wara normalise ci baat bu nekk ngir jéego bu nekk ci tàggat yaram - di yeex lool. Xayma buy wuutale bruit bi dafay soppi jafe-jafe bi ci xeetu xaaj bu ñaar: ñu jox benn mébet ak yenn misaali 'bruit' yuñ jëlee ci benn séddale buñ xamee, jàng ràññee misaalu dëgg bi ak bruit bi, loolu mooy indiwaat probabilite yiñ bëgg te duñu def benn normalisation bu leer. Échantillonnage négatif, bi xeetu skip-gram bu word2vec siiwal, cousin bu yomb la: ngir bépp dëgg (baatu, contexte) pair dafay sample k negatif ba noppi di tàggat model bi ngir jox poñ yu bari ci pair dëgg bi ak poñ yu néew ci fake yi, jëfandikoo ab mébet sigmoid. Ñoom ñaar ñépp dañuy soppi jafe-jafe bu seer ci klaas yu bari ci yeneen jafe-jafe yu yomb yu bari, loolu mooy tax tàggat yaram bu yaatu nekk lu am solo. Tanneef ci séddale bruit (unigram yu bari yu ñuy yéeg ba 3/4 doole) dafay am njeexital yu am solo ci kalite bi.
Gis-gis xarala
NCE dafay xayma benn model ci xaaj ay done ak bruit, te lu limu misaali bruit yi di gëna yokk, dafay jegesi maximum-likelihood ak softmax bu jaar yoon. Échantillonnage négatif dafay dindi bépp terme normalisation bu NCE, gëna xéewale log σ (poñ bu baax) + Σ log σ (−poñ bu baaxul). Loolu moo tax mu gëna gaaw waaye nekkatul estimatëru densité bu méngoo - dañu ko defar ngir jàng embeddings yu baax moo gën ñu kalibre probabilite yi. Sampling negatif ci distribution unigram bu nooy (frequency^0.75) dafay ekilibre kàddu yu bari ak yu bariwul.
njeextalu pexe
Njëgg ak budget
Dogal yi architecture di jël dañuy indi njariñ ak njëgu liggéey bi ay at ci ginaaw.
dogal yu gëna leer
Njàngalem xarala yi dafay jàppale ekip yi ñu tànn li gën, te baña yam ci li gëna bees daal.
Xool kalite
Tanneef yu gëna baax ci wàllu ingeñër dina wàññi jafe-jafe yi ci wàllu wóor ci liggéey bi.
Ëlëgu échantillonnage négatif ak xayma buy wuutale bruit
Xalaat bu am solo bi - jàng ci wuutale lu baax ak lu baaxul ci misaal - leegi dafay jàppale jàngu bu bees bi ñuy saytu sa bopp ak wuutale jàngat ci gis-gis, làkk, ak xalaat. Liggéey bi ci kanam dafay xoole ci mine negatif yu dëgër (tann negatif yu am xibaar ci barabu negatif yi), debiasing ngir negatif yu baaxul yi, ak scaling negatif yi ci njëg yu yomb jaaraleko ci bànku memory yu mag wala échantillonnage ci lots. Lu model yi di màgg, ay mébetu sampling yu baax dañuy wéy di am solo fépp fu bayaali génne yi wala ensemble kàndidaa yi yaatu, lu melni seetlu ak recommandé yu mag.
Doxal ci àdduna dëgg
word2vec skip-gram ak misaal bu baaxul di jàng ay baat ci ay miliyaar ciy jeton te amul softmax bu mat.
Royuwaayi làkk yi yàgg a jëfandikoo NCE ngir tàggat vocabulaire yu téemeeri junni kàddu ci anam wu jaar yoon.
Sistemu recommandé ak recuperation yi dañuy jël misaalu mbir yu baaxul yi jëfandikukat bi jëflantewul ak ñoom ngir tàggat xeetu embedding ñaari tour.
Graaf ak xam-xam-graf embeddings (lu melni, yàq boppu wala geen bu ñatti) jëfandikoo misaal yu baaxul ngir jàng diggante entite.
Risk yi ak balustrade yi
Optimize benn benchmark mën na nëbb ñakk kattan yu gëna yaatu ci sistem bi.
Njëg li ñuy fay ci infrastructure yi ak ci toppatoo dañuy faral di suufeel.
Bu sistem yi di gëna xawa jafee xam, jafe-jafe yi am ci wàllu kaaraange ak seetlu mën nañu gëna bari.
Roadmap ngir samp gi
Mandargal latency, kalite, ak njëg yi laata ngay jëfandikoo.
Benchmark ci biir sargal ak done yu dëggu.
Jumtukaay bi di saytu njuumte yi, derive bi ak njeextalu jëfandikukat bi.
Waajal rollback ak yooni tontu ci jafe-jafe yi laata ngay eskale.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Negative Sampling and Noise Contrastive Estimation quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Mine négatif en ligne et dur
Laaj yi ñuy faral di laaj
What is Negative Sampling and Noise Contrastive Estimation?
Échantillonnage négatif ak estimation contraste de bruit (NCE) ay pexe lañu yuy may model yi ñu jàng lu bari ci vocabulaire te duñu xayma softmax bu mat te seer lool. Duñu def poñ ci bépp génne bu mëna am, waaye dañu jàngal model bi wax misaal yu dëggu (positif) ci yenn misaal yu baaxul (negatif).
Ban jafe-jafe la échantillonnage négatif ak NCE njëkka saafara?
Ñari xeeti xam-xam yooyu moytu ñu normalise ci kaw vocabulaire bu bari ci reframing tàggat yaram ni discrimination binaire bu gëna xéewale.
naka la ko estimaasioŋ buy wuutale bruit di defaree liggéeyu jàng bi?
NCE dafay tàggat model bi ngir mëna wuutale misaal yi dëgg ak misaal yiñ jëlee ci distribution bruit buñ xamee.
Ban model moo siiwal sampling négatif ngir jàng embeddings baat?
Echantillonnage négatif bi ñu ngi ko dugal ba noppi siiwal ko ci xeetu skip-gram bu word2vec.
Lan moo wuutale échantillonnage négatif ak NCE bu mat?
Échantillonnage négatif dafay yombal NCE ci dindi normalisasioŋ, njaay njubte probabilistik ngir gaawaay ak dugal bu baax.
Lan moo waral ñuy faral di jël negatif yi ci distribution unigram bi yéeg ba 3/4 doole?
Exponent 0.75 dafay yombal séddaleb fréquence yi suko defee baat yiñ gëna xam duñu ëpp doole ba noppi baat yu bariwul ñuy feeñ.