Gumbel-Softmax da Reparameterization
Gumbel-Softmax wata dabara ce da ke ba da damar cibiyoyin sadarwar jijiyoyi 'samfuri' daga nau'ikan mabambanta yayin da har yanzu ana iya horar da su ta hanyar zuriya.
Dubawa
It matters because backpropagation normally can't flow through a random, discrete choice.
Zurfafa nutsewa
Cibiyoyin jijiyoyi suna koyo ta hanyar aika gradients baya ta kowane aiki. Amma ɗaukar nau'i mai ma'ana (kamar ɗaukar kalmar #7 na 50,000) tsalle ne mai wuya, wanda ba shi da bambanci, don haka gradients suna mutuwa a can. Dabarar sake fasalin yana sake rubuta samfurin bazuwar don haka bazuwar ya fito daga kafaffen amo na waje, yana barin hanya mai santsi, banbanta ga gradients. Gumbel-Softmax yana yin amfani da wannan ga masu canji masu ƙima: yana ƙara amo da aka rarraba Gumbel zuwa logits, sannan ya maye gurbin argmax mai ƙarfi tare da softmax mai sarrafa zafin jiki. A babban zafin jiki abin fitarwa yana da santsi mai laushi akan nau'ikan; yayin da zafin jiki ke faɗuwa zuwa sifili yana ƙaruwa zuwa kusa da vector mai zafi ɗaya, yana dawo da samfur na gaskiya yayin da ake iya bambanta gaba ɗaya.
Fahimtar Fasaha
Dabarar Gumbel-Max ta ce: ƙara ƙarar Gumbel (0,1) mai zaman kanta zuwa kowane logit da ɗaukar argmax yana haifar da ainihin samfurin daga rarraba softmax. Gumbel-Softmax yana musanya wancan mai wuyar argmax don softmax((log p + g)/tau). Matsakaicin zafin jiki yana shiga tsakanin santsi, babban rabe-raben entropy (babban tau) da kuma kusa-zafi ɗaya (karamin tau). Saboda an zana amo g a wajen hanyar sadarwa, hanyar daga logits zuwa fitarwa ta kasance mai bambanta.
Dabarun Tasiri
Kudin da kasafin kuɗi
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Shawarwari masu haske
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Kula da inganci
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Makomar Gumbel-Softmax da Reparameterization
Gumbel-Softmax ya kasance babban kayan aiki don bambance-bambancen latent, bincike daban-daban na gine-gine, ƙididdiga masu ƙididdigewa, da kuma koyan hanya a cikin tsarin cakuda-na-ƙwararru. Ana ci gaba da bincike akan ƙananan bambance-bambance, shakatawa na ƙasa-ƙasa (kamar Rao-Blackwellized da masu ƙididdigewa masu sarrafawa) da kuma kan jadawalin kashewa waɗanda ke daidaita ƙima na yanayin zafi da babban bambance-bambancen masu sanyi. Kamar yadda samfura ke ƙara yin fayyace yanke shawara, tsammanin waɗannan ci gaba da shakatawa za su kasance a tsakiya don yin irin waɗannan zaɓuɓɓukan da za a iya koyo daga ƙarshe zuwa ƙarshe.
Aiwatar da Gaskiyar Duniya
Horar da bambance-bambancen autoencoders tare da latent latent (masu hankali) maimakon na Gaussian masu ci gaba.
Binciken gine-gine daban-daban (misali, hanyoyin salon DARTS) zaɓin aikin da za a sanya a kowane Layer.
Koyan zaɓukan littafin code a cikin salon VQ da ƙirar wakilci mai hankali.
Shawarwari daban-daban na hanyar tuƙi ko yanke shawara a cikin cakuda-na-ƙwararru da cibiyoyin sadarwa na ƙididdigewa.
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Gumbel-Softmax and Reparameterization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Hanyoyin Sadarwar Maimaituwar Biyu
Tambayoyin da ake yawan yi
What is Gumbel-Softmax and Reparameterization?
Gumbel-Softmax wata dabara ce da ke ba da damar cibiyoyin sadarwar jijiyoyi 'samfuri' daga nau'ikan mabambanta yayin da har yanzu ana iya horar da su ta hanyar zuriya. Yana da mahimmanci saboda yada baya yawanci ba zai iya gudana ta hanyar bazuwar, zaɓi mai hankali ba.
Wace babbar matsala Gumbel-Softmax ke magance?
Samfura mai hankali ta hanyar argmax ba shi da bambanci, yana toshe gradients. Gumbel-Softmax yana ba da hutu mai ban sha'awa don haka har yanzu ana iya horar da cibiyar sadarwa daga ƙarshe zuwa ƙarshe.
A cikin dabarar gyarawa, daga ina bazuwar ta fito?
Sake daidaitawa yana motsa bazuwar zuwa madaidaicin amo mai zaman kansa, yana barin ƙayyadaddun ƙayyadaddun ayyuka na sigogin cibiyar sadarwa.
Dangane da dabarar Gumbel-Max, ta yaya za ku iya zana ainihin samfurin daga rarraba softmax?
Dabarar Gumbel-Max: argmax over (logits + i.d. Gumbel hayaniyar) an rarraba shi daidai a matsayin samfuri mai ƙima daga softmax na waɗannan logits.
Menene rawar ma'aunin zafin jiki (tau) a cikin Gumbel-Softmax?
Low tau yana tura softmax zuwa wani kusa da vector mai zafi ɗaya (kusa da samfurin gaskiya); high tau yana sa shi santsi da babban entropy. Yana cinikin kashe son zuciya da bambancin gradient.
Yayin da tau ke gabatowa sifili, kayan aikin Gumbel-Softmax na gabatowa menene?
Sanyaya zafin jiki zuwa sifili yana kaifafa softmax har sai ya kusan zaɓar nau'i guda ɗaya, yana dawo da halayen ƙira.