Ubuyobozi bwa tekiniki

Gumbel-Softmax na Reparameterisation

Gumbel-Softmax ni amayeri areka imiyoboro yimitsi 'sample' kuva mubyiciro bitandukanye mugihe ikiri gutozwa na gradient descente.

Umva iki gitaboAI-byavuzwe · ~2 min

Kuriyi page2 min soma
  1. Incamake
  2. Kwibira cyane
  3. Gushyira mu bikorwa Isi
  4. Kazoza ka Gumbel-Softmax na Reparameterisation
  5. Kubishyira mubikorwa
  6. Komeza Ubushakashatsi
  7. Ibibazo bikunze kubazwa

Incamake

Nibyingenzi kuko gusubira inyuma mubisanzwe ntibishobora gutembera muburyo butemewe, bwihariye.

Kwibira cyane

Imiyoboro ya neural yiga kohereza gradients inyuma muri buri gikorwa. Ariko gutoranya icyiciro cyihariye (nko gutoranya ijambo # 7 kuri 50.000) nugusimbuka gukomeye, kudatandukana, gradients zipfirayo. Amayeri ya reparameterisation yongeye kwandika icyitegererezo kuburyo butunguranye buturuka kumasoko ahamye yo hanze, hasigara inzira yoroshye, itandukanye ya gradients. Gumbel-Softmax ikoresha ibi muburyo butandukanye: yongeraho urusaku rwakwirakwijwe na Gumbel kuri logi, hanyuma igasimbuza argmax ikomeye hamwe na softmax igenzurwa n'ubushyuhe. Ku bushyuhe bwo hejuru ibisohoka ni blob yoroshye hejuru y'ibyiciro; nkuko ubushyuhe bugabanuka kuri zeru bikarishye hafi ya vector imwe ishyushye, igarura icyitegererezo nyacyo mugihe igumye itandukanye muri rusange.

Gushyira mu bikorwa Isi

Guhugura autoencoders zitandukanye hamwe na code (catretique) code yihishe aho kuba iyikomeza ya Gaussian gusa.

Itandukaniro ritandukanye ryubwubatsi bwubushakashatsi (urugero, uburyo bwa DARTS-uburyo) guhitamo igikorwa cyo gushyira kuri buri cyiciro.

Kwiga gutandukanya codebook yatoranijwe muburyo bwa VQ na moderi yo kwerekana.

Gutandukanya inzira zitandukanye cyangwa gufata ibyemezo muruvange-rwinzobere nimbuga-yo kubara.

Kazoza ka Gumbel-Softmax na Reparameterisation

Gumbel-Softmax ikomeza kuba igikoresho cyambere cyo gutandukanya ibintu byihishe, gushakisha imyubakire itandukanye, kwerekana imiterere ya vector, hamwe no kwiga inzira muri mix-ya-sisitemu. Ubushakashatsi burakomeza kuri vari-variance, hasi-kubogama kuruhuka (nka Rao-Blackwellized na control-variate igereranya) no kuri gahunda ya annealing igereranya kubogama kwubushyuhe bukabije hamwe nubushyuhe bukabije bwimbeho. Nkuko abanyamideli bagenda bafata ibyemezo byihariye, tegereza ibyo kuruhuka bikomeza kuguma hagati kugirango uhitemo amahitamo nkaya yiga-iherezo.

Kubishyira mubikorwa

Impamvu ari ngombwa

  • Igiciro na bije. Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
  • Ibyemezo bisobanutse. Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
  • Kugenzura ubuziranenge. Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.

Witondere

  • Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
  • Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
  • Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.

Intambwe ikurikira

  1. Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
  2. Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
  3. Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
  4. Tegura inzira yo gusubiza ibyabaye mbere yo gupima.

Komeza Ubushakashatsi

Ibaruwa y'amakuru y'ubuntu

Ibuka amakuru ya buri munsi ya AI

Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.

Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.

Isuzume

Take the Gumbel-Softmax and Reparameterization quiz

Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.

Tangira ikibazo

Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano

Ibibazo bikunze kubazwa

Gumbel-Softmax na Reparameterisation ni iki?

Gumbel-Softmax ni amayeri areka imiyoboro yimitsi 'sample' kuva mubyiciro bitandukanye mugihe ikiri gutozwa na gradient descente. Nibyingenzi kuko gusubira inyuma mubisanzwe ntibishobora gutembera muburyo butemewe, bwihariye.

Why is a categorical sample difficult to train through with ordinary backpropagation?

The guide explains gradients do not pass through a hard discrete jump.

What does the reparameterization trick aim to do?

The guide describes moving randomness into a noise source and preserving a smooth path.

How does Gumbel-Softmax approximate a categorical choice?

The guide gives the noisy softmax construction for differentiable categorical sampling.

What happens to the distribution as temperature becomes large?

High temperature yields a smoother distribution over categories.

What happens as temperature approaches zero?

The guide says low temperature sharpens toward a near one-hot vector.