Gumbel-Softmax na Reparameterisation
Gumbel-Softmax ni amayeri areka imiyoboro yimitsi 'sample' kuva mubyiciro bitandukanye mugihe ikiri gutozwa na gradient descente.
Incamake
It matters because backpropagation normally can't flow through a random, discrete choice.
Kwibira cyane
Imiyoboro ya neural yiga kohereza gradients inyuma muri buri gikorwa. Ariko gutoranya icyiciro cyihariye (nko gutoranya ijambo # 7 kuri 50.000) nugusimbuka gukomeye, kudatandukana, gradients zipfirayo. Amayeri ya reparameterisation yongeye kwandika icyitegererezo kuburyo butunguranye buturuka kumasoko ahamye yo hanze, hasigara inzira yoroshye, itandukanye ya gradients. Gumbel-Softmax ikoresha ibi muburyo butandukanye: yongeraho urusaku rwakwirakwijwe na Gumbel kuri logi, hanyuma igasimbuza argmax ikomeye hamwe na softmax igenzurwa n'ubushyuhe. Ku bushyuhe bwo hejuru ibisohoka ni blob yoroshye hejuru y'ibyiciro; nkuko ubushyuhe bugabanuka kuri zeru bikarishye hafi ya vector imwe ishyushye, igarura icyitegererezo nyacyo mugihe igumye itandukanye muri rusange.
Ubushishozi
Amayeri ya Gumbel-Max agira ati: kongeramo urusaku rwigenga rwa Gumbel (0,1) kuri buri logit no gufata argmax bitanga urugero nyarwo ruva muri softmax. Gumbel-Softmax ihinduranya argmax ikomeye kuri softmax ((log p + g) / tau). Ubushyuhe bwa tau buhuza hagati yikwirakwizwa ryoroshye, ryinshi rya entropiya (tau nini) hamwe na discret imwe-ishyushye (tau nto). Kuberako urusaku g rwatoranijwe hanze y'urusobe, inzira iva mumyandikire isohoka iguma itandukanye.
Ingaruka z'Ingamba
Igiciro na bije
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo bisobanutse
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Kugenzura ubuziranenge
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Kazoza ka Gumbel-Softmax na Reparameterisation
Gumbel-Softmax ikomeza kuba igikoresho cyambere cyo gutandukanya ibintu byihishe, gushakisha imyubakire itandukanye, kwerekana imiterere ya vector, hamwe no kwiga inzira muri mix-ya-sisitemu. Ubushakashatsi burakomeza kuri vari-variance, hasi-kubogama kuruhuka (nka Rao-Blackwellized na control-variate igereranya) no kuri gahunda ya annealing igereranya kubogama kwubushyuhe bukabije hamwe nubushyuhe bukabije bwimbeho. Nkuko abanyamideli bagenda bafata ibyemezo byihariye, tegereza ibyo kuruhuka bikomeza kuguma hagati kugirango uhitemo amahitamo nkaya yiga-iherezo.
Gushyira mu bikorwa Isi
Guhugura autoencoders zitandukanye hamwe na code (catretique) code yihishe aho kuba iyikomeza ya Gaussian gusa.
Itandukaniro ritandukanye ryubwubatsi bwubushakashatsi (urugero, uburyo bwa DARTS-uburyo) guhitamo igikorwa cyo gushyira kuri buri cyiciro.
Kwiga gutandukanya codebook yatoranijwe muburyo bwa VQ na moderi yo kwerekana.
Gutandukanya inzira zitandukanye cyangwa gufata ibyemezo muruvange-rwinzobere nimbuga-yo kubara.
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Gumbel-Softmax and Reparameterization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Imiyoboro ibiri isubiramo
Ibibazo bikunze kubazwa
What is Gumbel-Softmax and Reparameterization?
Gumbel-Softmax ni amayeri areka imiyoboro yimitsi 'sample' kuva mubyiciro bitandukanye mugihe ikiri gutozwa na gradient descente. Nibyingenzi kuko gusubira inyuma mubisanzwe ntibishobora gutembera muburyo butemewe, bwihariye.
Ni ikihe kibazo cy'ibanze Gumbel-Softmax ikemura?
Gutoranya neza ukoresheje argmax ntaho bitandukaniye, guhagarika gradients. Gumbel-Softmax itanga uburuhukiro butandukanye kuburyo umuyoboro ushobora gutozwa amaherezo-kurangira.
Mumayeri ya reparameterisation, guhitamo biva he?
Gusubiramo ibintu byimura urusaku rwigenga rwigenga, hasigara imikorere, itandukanye yibikorwa byurusobe.
Ukurikije amayeri ya Gumbel-Max, nigute ushobora gushushanya icyitegererezo nyacyo mugukwirakwiza softmax?
Amayeri ya Gumbel-Max: argmax hejuru (logits + i.i.d. urusaku rwa Gumbel) yatanzwe neza nkurugero rwicyiciro cyavuye kuri softmax yizo logi.
Ni uruhe ruhare rw'ubushyuhe (tau) muri Gumbel-Softmax?
Tau yo hasi isunika softmax yerekeza hafi ya vector imwe ishyushye (hafi y'icyitegererezo nyacyo); muremure tau ituma byoroha kandi-entropiya. Igurisha kubogama kubitandukanya buhoro buhoro.
Mugihe tau yegereye zeru, Gumbel-Softmax isohoka yegera iki?
Gukonjesha ubushyuhe bugana kuri zeru bikarisha softmax kugeza igihe ihitamo icyiciro kimwe, igarura imyitwarire idasanzwe.