Kugereranya nabi no gusakuza kugereranya
Gutoranya nabi hamwe no Kugereranya Urusaku (NCE) ni amayeri areka abanyamideli biga hejuru yamagambo manini batabanje kubara software yuzuye ihenze.
Incamake
Aho gutsinda amanota yose ashoboka, bigisha icyitegererezo kuvuga ingero zifatika (nziza) zivuye mubitigiri byimpimbano (bibi).
Kwibira cyane
Iyo amagambo afite amagambo ibihumbi magana, softmax isanzwe igomba guhinduka kuri buri jambo kuri buri ntambwe yo gutoza - buhoro cyane. Urusaku rutandukanye rwo kugereranya rusubiramo ikibazo nkuburyo bubiri: ukurikije intego hamwe n '' urusaku 'ruto rwakuwe mubisaranganywa bizwi, wige gutandukanya icyitegererezo nyacyo n urusaku, rusubizamo byimazeyo ibyifuzo byateganijwe bitari bisanzwe. Icyitegererezo kibi, cyamamaye kubijyanye na skip-gram yerekana ijambo2vec, ni mubyara woroheje: kuri buri kintu cyukuri (ijambo, imiterere) byombi byerekana urugero k ibibi kandi bigatoza icyitegererezo cyo gutanga amanota menshi kubantu nyabo hamwe n amanota make kuri feke, ukoresheje intego ya sigmoid. Byombi bihindura ikibazo gihenze cyibyiciro byinshi muburyo bubiri buhendutse, bigatuma amahugurwa manini yo gushira mubikorwa. Guhitamo gukwirakwiza urusaku (akenshi unigram yazamutse kuri 3/4 imbaraga) bigira ingaruka nziza kubwiza.
Ubushishozi
NCE igereranya icyitegererezo mugutondekanya amakuru n urusaku, kandi uko umubare wurusaku rwurusaku rwiyongera birashoboka cyane ko bishoboka cyane hamwe na softmax ikwiye. Gutoranya nabi bigabanya NCE amagambo asanzwe rwose, guhuza log σ (amanota meza) + Σ log σ (scorenegative score). Ibyo bituma byihuta ariko ntibikiri igereranya ryubucucike - byateguwe kugirango wige gushiramo neza aho guhitamo amahirwe. Gutoranya ibibi biva mugukwirakwiza unigram yoroshye (inshuro ^ 0,75) iringaniza amagambo asanzwe kandi adasanzwe.
Ingaruka z'Ingamba
Igiciro na bije
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo bisobanutse
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Kugenzura ubuziranenge
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Ejo hazaza h'icyitegererezo kibi no kugereranya urusaku
Igitekerezo cyibanze - wige gutandukanya ibyiza n'ibibi byiteganijwe - ubungubu ushimangira uburyo bugezweho bwo kwiyobora no kugereranya kwigira mubyerekezo, ururimi, hamwe nibyifuzo. Ibikorwa bizaza byibanda ku bucukuzi bw'amabuye y'agaciro (guhitamo ibibi bitanga amakuru aho guhitamo), gutesha agaciro ibibi, no gupima ibibi bihendutse binyuze muri banki nini zo kwibuka cyangwa mu cyitegererezo. Mugihe icyitegererezo gikura, intego zicyitegererezo zikomeza kuba ingenzi ahantu hose hasohoka umwanya cyangwa abakandida bashizeho ni nini cyane, nko kugarura no gutanga ibyifuzo binini.
Gushyira mu bikorwa Isi
ijambo2vec gusimbuka-garama hamwe nicyitegererezo cyiza cyo kwiga ijambo gushiramo kuva miriyari ya tokens idafite softmax yuzuye.
Ururimi rwicyitegererezo mu mateka ukoresheje NCE kugirango uhugure hejuru yamagambo ibihumbi magana yamagambo neza.
Sisitemu yo kugarura no kugarura ibintu byerekana 'ibintu bibi' umukoresha atigeze akorana nogutoza iminara ibiri yo gushyiramo iminara.
Igishushanyo n'ubumenyi-bishushanya (urugero, kwangiza umutwe wa gatatu cyangwa umurizo) ukoresheje ingero mbi kugirango wige umubano wibintu.
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Negative Sampling and Noise Contrastive Estimation quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Kumucukuzi kumurongo kandi bikomeye
Ibibazo bikunze kubazwa
Ikigereranyo Cyiza no Kugereranya Ikigereranyo Niki?
Gutoranya nabi hamwe no Kugereranya Urusaku (NCE) ni amayeri areka abanyamideli biga hejuru yamagambo manini batabanje kubara software yuzuye ihenze. Aho gutsinda amanota yose ashoboka, bigisha icyitegererezo kuvuga ingero zifatika (nziza) ziva mubitigiri byimpimbano (bibi).
Ni ikihe kibazo icyitegererezo cyiza na NCE gikemura cyane?
Ubwo buryo bwombi bwirinda guhuza amagambo menshi muguhindura imyitozo nkivangura rihendutse.
Nigute Urusaku rutandukanye rwo kugereranya rusubiramo imirimo yo kwiga?
NCE itoza icyitegererezo cyo gutandukanya ingero nyazo ziteganijwe kuva urusaku ruzwi.
Niyihe moderi yakwirakwije icyitegererezo cyiza cyo kwiga gushyiramo ijambo?
Icyitegererezo cyiza cyatangijwe kandi gikundwa na skip-gram variant ya word2vec.
Ni gute icyitegererezo kibi gitandukanye na NCE yuzuye?
Icyitegererezo cyiza cyoroshya NCE mukuraho ibisanzwe, gucuruza bishoboka gukosora umuvuduko no gushiramo neza.
Ni ukubera iki ibibi byakunze gutangwa kuva kugabana unigram kuzamurwa kuri 3/4 imbaraga?
0,75 ibyerekana byorohereza gukwirakwiza inshuro kuburyo amagambo asanzwe atiganje kandi amagambo adasanzwe aracyagaragara.