Gumbel-Softmax ak defarwaat
Gumbel-Softmax ab pexe la buy may reso neuronal yi ñu 'sample' ci kategori yu wuute te ba leegi mën nañu leen tàggat ci wàccinu gradient.
Résumé
Dafa am solo ndax backpropagation mënul jaar ci tànneef bu bari te wuute.
Plongeur bu xóot
Reseau neuronal yi dañuy jàng ci yónnee gradient yi ci ginaaw ci bépp liggéey. Waaye jël misaal ci kategori bu wuute (lu melni tànn baat #7 ci 50,000) lu jafe la, mënul wuute, kon gradient yi dañuy dee foofu. Reparametrization trick bi dafay binndaat échantillonnage aleatoire suko defee aleatoire bi joge ci ab balluwaay bu amul benn bruit bu nekk ci biti, bàyyi yoon wu nooy, wuñu mëna wuutale ci gradient yi. Gumbel-Softmax dafay jëfandikoo lii ci variable yiñ tànn: dafay yokk bruit bu Gumbel séddale ci logit yi, ba noppi mu wecci argmax bu dëgër bi ak softmax buñ kontre tàngoor wi. Su tàngoor wi yéegee, génne gi dafay nuru lu nooy ci kaw kategori yi; Bi tàngoor wi wàccee ba ci zero, dafay gëna ñaw ba ci vecteur bu jege benn tàng, di am sampling bu dëggu bi, di wéy di wuute ci biir.
Gis-gis xarala
Gumbel-Max dafay wax: yokk Gumbel (0,1) bruit bu moom boppam ci logit bu nekk nga jël argmax bi dafay joxe misaal bu dëggu ci séddaleb softmax bi. Gumbel-softmax dafay wecci argmax bu dëgër bi ak softmax ((log p + g)/tau). Tàngoor tau dafay jaxasoo ci digganté distribution bu nooy te bari entropi (tau bu rëy) ak benn tàngoor bu jege diskret (tau bu ndaw). Ndax bruit g dañu ko sample ci biti reso bi, yoon wi diggante logits ak output dafay wéy di wuute.
njeextalu pexe
Njëgg ak budget
Dogal yi architecture di jël dañuy indi njariñ ak njëgu liggéey bi ay at ci ginaaw.
dogal yu gëna leer
Njàngalem xarala yi dafay jàppale ekip yi ñu tànn li gën, te baña yam ci li gëna bees daal.
Xool kalite
Tanneef yu gëna baax ci wàllu ingeñër dina wàññi jafe-jafe yi ci wàllu wóor ci liggéey bi.
Ëlëgu Gumbel-Softmax ak defarwaat
Gumbel-Softmax mingi wéy di nekk jumtukaay buñ jagleel ay variable yu nëbbu, seetlu architecture buñ mëna wuutale, model yuñ xayma ci vecteur, ak jàng yoon ci sistem yu jaxasoo ak kàngam. Gëstu baa ngi wéy ci wàll wi gëna néew variance, gëna néew biais (lu melni Rao-Blackwellized ak estimatëri variate kontrôle) ak ci oraaru annealing yuy ekilibre biais tàngoor wu tàng wi ak variance gradient bu kawe bi ci sedd yi. Kom model yi dañuy gëna jël yenn dogal yu leer te wuute, xaarandil yii relaxations yuy wéy di des ci diggu def tànneef yu mel noonu ñu mëna jàng ci njeexte ba ci njeexte.
Doxal ci àdduna dëgg
Taggat autoencodeur yu wuute ak kode yu nëbbu yu kategorik (diskret) ci barabu kode Gaussien yu wéy kese.
Seetug architecture neuronal bu wuute (lu melni, pexe yu nuroo ak DARTS) di tànn ban liggéey lañu wara def ci bu nekk ci couche yi.
Jàng tànneefi téere kode yu wuute ci xeetu VQ ak xeetu wane yu wuute.
Routing bu wuute wala dogal ci gating ci njaxasu-ekspert ak reso ordinatër yu am sart.
Risk yi ak balustrade yi
Optimize benn benchmark mën na nëbb ñakk kattan yu gëna yaatu ci sistem bi.
Njëg li ñuy fay ci infrastructure yi ak ci toppatoo dañuy faral di suufeel.
Bu sistem yi di gëna xawa jafee xam, jafe-jafe yi am ci wàllu kaaraange ak seetlu mën nañu gëna bari.
Roadmap ngir samp gi
Mandargal latency, kalite, ak njëg yi laata ngay jëfandikoo.
Benchmark ci biir sargal ak done yu dëggu.
Jumtukaay bi di saytu njuumte yi, derive bi ak njeextalu jëfandikukat bi.
Waajal rollback ak yooni tontu ci jafe-jafe yi laata ngay eskale.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Gumbel-Softmax and Reparameterization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Reseau yu am ñaari yoon
Laaj yi ñuy faral di laaj
Luy Gumbel-Softmax ak Reparametraasioŋ?
Gumbel-Softmax ab pexe la buy may reso neuronal yi ñu 'sample' ci kategori yu wuute te ba leegi mën nañu leen tàggat ci wàccinu gradient. Dafa am solo ndax backpropagation mënul jaar ci tànneef bu bari te wuute.
Lan mooy jafe-jafe bi Gumbel-Softmax mëna saafara?
Échantillonnage diskret jaaraleko ci argmax mënul wuutale, dafay tere gradient yi. Gumbel-Softmax dafay joxe ab féexal bu wuute suko defee ñu mëna tàggat reso bi ba leegi.
Ci kaf reparametrisasioŋ bi, fan la aleatoire bi bawoo?
Reparametrizaasioŋ dafay toxal aleatoire bi ci benn variable bruit bu moom boppam, bàyyi fa fonction deterministe, differansier ci parametru reso bi.
Buñu sukkandikoo ci kaf Gumbel-Max, naka ngay mëna jëlee misaal bu dëggu ci séddaleb softmax?
Gumbel-Max trick: argmax ci kaw (logits + maanaam bruit Gumbel) dañu koy séddale ndànk ni misaal bu jëm ci softmax logits yooyu.
Lan mooy wareefu tàngoor (tau) ci Gumbel-Softmax?
Tau bu woyof dafay puus softmax ci vecteur bu jege benn tàng (jege sampling dëgg); tau bu bari daf koy def lu nooy te am entropi bu bari. Dafay xajamal njaaxaanaay ak faraasu gradient.
Bi tau jegee nul, Gumbel-Softmax génne jegesi lan?
Seddal tàngoor wi ci nul dafay ñaw softmax ba keroog muy tànnee benn kategori, defaraat jeffin sampling diskret.