InoteveraGaidhi rinotevera
Softmax Regression yeMulticlass Classification
Tekinoroji
Nhungamiro yehunyanzvi
Gumbel-Softmax uye Reparameterization
Gumbel-Softmax idhiri rinoita kuti neural network 'sample' kubva kune discrete mapoka ichiri kudzidziswa ne gradient descent.
Teerera kugwaro iriAI-yakarondedzerwa · ~2 min
Pa peji ino2 min verenga
Pfupiso
Izvo zvine basa nekuti backpropagation kazhinji haigone kuyerera kuburikidza nechero, discrete sarudzo.
Kudzika Kwakadzika
Neural network inodzidza nekutumira gradients kumashure kuburikidza nekushanda kwese. Asi sampling chikamu chakasarudzika (sekunhonga izwi # 7 pa50,000) yakaoma, isingasiyanise kusvetuka, saka magradients anofira ipapo. Iro reparameterization trick inonyora zvekare sampling isina kujairika kuitira kuti kusarongeka kuuye kubva kune yakamisikidzwa yekunze ruzha sosi, ichisiya yakatsetseka, inosiyanisa nzira yemagradients. Gumbel-Softmax inoshandisa izvi kumhando dzakasiyana-siyana: inowedzera Gumbel-yakaparadzirwa ruzha kune zvinyorwa, yobva yatsiva iyo yakaoma argmax ine tembiricha-inodzorwa softmax. Pakupisa kwepamusoro kubudiswa kunoputika kwakatsetseka pamusoro pezvikamu; sezvo tembiricha inodonha yakananga ku zero inorodza yakananga kune imwe-inopisa vector, kudzoreredza sampling chaiyo ichiramba ichisiyaniswa kwese.
Kuitwa muNyika yeChokwadi
Kudzidzira akasiyana-siyana autoencoder ane categorical (discrete) latent macode pane anongoenderera eGaussian.
Differentiable neural architecture search (semuenzaniso, DARTS-style nzira) kusarudza kuti ndeipi mashandiro ekuisa pane imwe neimwe layer.
Kudzidza discrete codebook sarudzo muVQ-maitiro uye discrete anomiririra modhi.
Yakasiyana-siyana nzira kana gating sarudzo mumusanganiswa-we-nyanzvi uye mamiriro-computation network.
Ramangwana reGumbel-Softmax uye Reparameterization
Gumbel-Softmax inoramba iri chigadziriso chekushandisa che discrete latent variables, inosiyaniswa yekutsvaga yekuvaka, vector-quantized modhi, uye yakadzidza nzira musanganiswa-we-nyanzvi masisitimu. Tsvagiridzo inoenderera pane yakaderera-mutsauko, yakaderera-yakarerekera kuzorora (seRao-Blackwellized uye control-variate estimators) uye pane annealing marongero anodzikamisa kurerekera kwekudziya kwekudziya kupesana nepamusoro gradient musiyano weanotonhora. Sezvo mamodheru achiwedzera kuita sarudzo dzakajeka, tarisira kuti idzi zororo dzinoramba dzichigara pakati pekuita sarudzo dzakadai kudzidzira kupera-kumagumo.
Kuzvishandisa
Nei zvichikosha
- Mutengo uye bhajeti. Zvisarudzo zvezvivakwa zvinotyaira kuita uye mutengo wekushandisa kwemakore.
- Sarudzo dzakajeka. Dzidzo yehunyanzvi inobatsira zvikwata kusarudza murwi wakakodzera, kwete iwo mutsva chete.
- Kudzora kwemhando yepamusoro. Sarudzo dzeinjiniya dziri nani dzinoderedza zviitiko zvekuvimbika mukugadzira.
Chenjera
- Kugadzirisa imwe bhenji kunogona kuvanza yakafara system kushaya simba.
- Infrastructure uye mari yekugadzirisa inowanzotarisirwa pasi.
- Chengetedzo uye kucherechedzwa mapundu anogona kukura sezvo masisitimu anowedzera kuoma.
Matanho anotevera
- Tsanangura latency, mhando, uye mutengo zvinangwa usati waitwa.
- Benchmark pasi pechokwadi mutoro uye data mamiriro.
- Chishandiso chekutarisa zvikanganiso, kudonha, uye mushandisi maitiro.
- Gadzirira nzira dzekudzosera kumashure uye dzezviitiko usati wawedzera.
Ramba Uchiongorora
Tsamba yemahara
Tora iyo yemazuva ese AI pfupiso
Nhatu dzakasimbiswa dzeAI nyaya mangwanani ega ega pakati pevhiki, dzakanyorwa neChirungu chakajeka. Yemahara zvachose, hapana ads.
Imwe email zuva rega rega revhiki. Bvisa kunyoresa kamwe chete. Hatimbotengese kana kugovera kero yako.
Zviedze iwe pachako
Take the Gumbel-Softmax and Reparameterization quiz
Mhinduro yepakarepo pamhinduro yega yega, uye chitupa chinogovaniswa chine ID inobatika kana wapedza kosi.
Tsigira dzidzo yemahara yeAI. AI Understanding ndeye 501(c)(3) isingabatsiri - hapana ads, hapana paywall, zvachose. Ita chipo
Mibvunzo inowanzo bvunzwa
Chii chinonzi Gumbel-Softmax uye Reparameterization?
Gumbel-Softmax idhiri rinoita kuti neural network 'sample' kubva kune discrete mapoka ichiri kudzidziswa ne gradient descent. Izvo zvine basa nekuti backpropagation kazhinji haigone kuyerera kuburikidza nekusarudzika, discrete sarudzo.
Why is a categorical sample difficult to train through with ordinary backpropagation?
The guide explains gradients do not pass through a hard discrete jump.
What does the reparameterization trick aim to do?
The guide describes moving randomness into a noise source and preserving a smooth path.
How does Gumbel-Softmax approximate a categorical choice?
The guide gives the noisy softmax construction for differentiable categorical sampling.
What happens to the distribution as temperature becomes large?
High temperature yields a smoother distribution over categories.
What happens as temperature approaches zero?
The guide says low temperature sharpens toward a near one-hot vector.
Ramba uchidzidza
Mazano anoenderana
Mamwe madhairekitori akasarudzirwa nyaya iyi