Gahunda yo Gutoranya no Kugaragaza Kubogama
Kubogama kubogamye ni icyuho kigaragara mugihe icyitegererezo cyatojwe gusa prefixes nziza kigomba, kubitekerezo, imiterere kubisubizo byayo bidatunganye.
Incamake
Scheduled sampling is a curriculum that gradually closes that gap.
Kwibira cyane
Abanyamideli bahuguwe hamwe nabarimu bahatira gusa kubona ibimenyetso-byukuri nkibimenyetso, ariko mugihe cyibisekuruza basubiza ibyo bahanuye. Iyo ikosa ryambere ryerekanye icyitegererezo muri leta itigeze ihura nayo mugihe cyamahugurwa, amakosa arashobora gukina urubura, uburyo bwo gutsindwa bwitwa bias bias. Icyitegererezo cyateganijwe, cyatangijwe na Bengio na bagenzi be muri 2015, gikemura iki kibazo cyo guhanagura igiceri kuri buri ntambwe ya decoding mugihe cy'amahugurwa: hamwe nibishoboka byose bigaburira ikimenyetso nyacyo (guhatira abarimu) naho ubundi kigaburira icyitegererezo icyitegererezo. Amahirwe yo gukoresha ukuri kwibanze atangirira hafi imwe kandi akangirika kumahugurwa akoresheje ingengabihe (umurongo, yerekanwe, cyangwa invers-sigmoid), bityo moderi igenda ihura nibisubizo byayo kandi ikiga gukira amakosa yayo.
Ubushishozi
Kuri intambwe t icyitegererezo cyerekana Bernoulli ihinduka hamwe na epsilon_i yo guhitamo ikimenyetso cya zahabu; epsilon_i ibora uko imyitozo ikomeza. Ubwitonzi ni uko kugaburira ibimenyetso byatoranijwe bituma intego ibogama hamwe na discret sampling idatandukanijwe, bityo gradients ntizitemba neza binyuze mukimenyetso cyagabanijwe. Impinduka zikoresha neza-zinyuze muri Gumbel-softmax cyangwa kuruhuka gutandukanye kugirango ugabanye ibi, kandi uburyo-urwego rukurikirana bihindura ibipimo nka BLEU muburyo butaziguye.
Ingaruka z'Ingamba
Igiciro na bije
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo bisobanutse
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Kugenzura ubuziranenge
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Ejo hazaza hateganijwe gutoranya no kubogama
Kururimi runini rwa Transformer yerekana ingaruka zifatika zo kubogama kubogama, kuva amakuru manini nubunini bigabanya, hamwe nuburyo nka RLHF ivugurura imyitwarire yibisekuruza bitaziguye. Nubwo bimeze bityo, icyitegererezo cyateganijwe hamwe nababakomokaho gikomeza kuba ingirakamaro kuri moderi ntoya, ibisekuruza byubatswe, hamwe nibikorwa bikenewe neza. Akazi kazoza gahuza integanyanyigisho, intego-yo gushimangira-intego zikurikirana, hamwe namahugurwa-yingaruka-nto kugirango ahuze uburyo moderi yatojwe nuburyo yangirika.
Gushyira mu bikorwa Isi
Guhugura ishusho-yerekana icyitegererezo hamwe nicyitegererezo cyateganijwe kugirango yige gukomeza neza nyuma yijambo ryahanuwe ridatunganye
Kwangiriza umwarimu-guhatira amahirwe hamwe na gahunda ihindagurika-sigmoid muri sisitemu yo guhindura imashini
Gupima ikiganiro kigenda kijya mumuzingo udahuye nkikimenyetso-kibogamye kimenyetso cya mwarimu mwiza uhatira
Ugereranije amanota ya BLEU yincamake yatojwe hamwe numwarimu wuzuye uhatira hamwe nuwahuguwe hamwe nicyitegererezo
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Scheduled Sampling and Exposure Bias quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Kugereranya nabi no gusakuza kugereranya
Ibibazo bikunze kubazwa
What is Scheduled Sampling and Exposure Bias?
Kubogama kubogamye ni icyuho kigaragara mugihe icyitegererezo cyatojwe gusa prefixes nziza kigomba, kubitekerezo, imiterere kubisubizo byayo bidatunganye. Gahunda yo gutoranya ni integanyanyigisho igenda ifunga buhoro buhoro icyo cyuho.
Icyitegererezo cyateganijwe gikora iki mugihe cy'amahugurwa?
Gahunda y'icyitegererezo itondekanya ivanga ubutaka-ukuri hamwe nicyitegererezo cyatanzwe nicyitegererezo nka decoder yinjiza, igenzurwa no kubora.
Nigute bishoboka ko ukoresha ibimenyetso-byukuri byerekana ibimenyetso hejuru y'amahugurwa ateganijwe?
Gahunda itangira hafi yumwarimu wuzuye guhatira no kubora kuburyo icyitegererezo kigenda gihura nubuhanuzi bwacyo uko gitera imbere.
Ninde watangije icyitegererezo, kandi hafi ryari?
Icyitegererezo cyateganijwe cyatanzwe na Samy Bengio na bagenzi be muri 2015 kugirango bahanure neza hamwe nimbuga zisubirwamo.
Ni ubuhe buryo bw'ingengabihe bukunze kuvugwa kubora umwarimu uhatira?
Igikorwa cyumwimerere cyasabye umurongo, icyerekezo, na invers-sigmoid gahunda yo kubora kugirango ugabanye ubutaka-ukuri gutoranya bishoboka.