Xiga xigaHagaha xiga
Softmax Regression for Multiclassification
Farsamo
HAGAHA Farsamada
Gumbel-Softmax iyo Dib u habeyn
Gumbel-Softmax waa khiyaamo u sahlaysa shabakadaha neural'ka 'muunad' qaybo kala duwan iyadoo weli lagu tababari karo faracyo hoose.
Dhageyso hagahanAI-waa laga sheekeeyay · ~2 min
Boggaan2 daqiiqo akhri
Dulmar
It matters because backpropagation normally can't flow through a random, discrete choice.
quusid qoto dheer
Shabakadaha neerfaha waxay wax ku bartaan iyagoo dib u soo diraya gradients qalliin kasta. Laakin qaadashada qayb gaar ah (sida xulashada #7 ee 50,000) waa boodi adag, oo aan kala duwanayn, markaa gradients halkaas ayay ku dhintaan. Xeeladda dib-u-soo-kabashada waxay dib u qortaa muunad random-ka ah si ay kala-sooc la'aantu uga timaad il qaylo dibadda ah oo go'an, taasoo ka tagaysa waddo siman, oo kala duwanaansho leh ee jaangooyooyinka. Gumbel-Softmax waxa ay ku dabaqdaa doorsoomayaasha kala duwan: waxa ay ku darsataa sawaxanka Gumbel qaybsan ee sawaxanka, ka dibna waxa ay ku bedeshaa argmax adag oo leh softmax heerkulku kantaroolo. Heerkulka sare wax soo saarku waa barar siman oo ka sarreeya qaybaha; Marka heerkulku hoos u dhaco ee eber waxa uu soo afaysanayaa xagal hal-kulul u dhaw, ka soo kabsashada muunad run ah iyada oo la sii kala duwanaan karo.
Dhaqangelinta Adduunka-dhabta ah
Tababarka codbixiyeyaasha kala duwan ee autoencoders oo wata koodh qarsoon oo gaar ah (discrete) halkii aad ka ahaan lahayd kuwa Gaussian ah oo joogto ah.
Raadinta qaab dhismeedka neerfaha ee kala duwan (tusaale, hababka DARTS-style) doorashada hawlgalka lakab kasta.
Barashada xulashooyinka buug-qodeedka gaarka ah ee qaabka VQ-ga iyo moodooyinka matalaadda gaarka ah.
Go'aannada marin-wareejinta ama gating-ka ee kala duwan ee khubarada-isku-dhafka ah iyo shabakadaha xisaabinta shuruudaha.
Mustaqbalka Gumbel-Softmax iyo Dib-u-habaynta
Gumbel-Softmax ayaa weli ah aaladda aasaasiga ah ee doorsoomayaal daahsoon oo kala duwan, raadinta qaab dhismeedka la kala duwanaan karo, moodooyinka la qiyaasey, iyo habaynta la bartay ee nidaamyada isku-dhafka ah ee khubarada. Cilmi-baadhistu waxay ku sii socotaa kala duwanaanshiyaha hoose, nasashada eexda hoose (sida Rao-Blackwellized and control-variate spectimators) iyo jadwalka nuugista ee dheelli tiran eexda heerkulku kulushahay kala duwanaanshaha sare ee qabowga. Marka moodellada ay sii kordhayaan go'aamo qeexan, filo nasashadan joogtada ah inay udub dhexaad u noqdaan samaynta doorashooyinkan la baran karo dhamaadka-ilaa-dhamaadka.
Ku dhaqangelinta
Maxay muhiim u tahay
- Qiimaha iyo miisaaniyada. Go'aamada qaab-dhismeedku waxay horseedaan waxqabadka iyo kharashka hawlgalka sannadaha.
- Go'aamo cad. Waxbarashada farsamada waxay ka caawisaa kooxaha inay doortaan xidhmo sax ah, ma aha oo kaliya kan ugu cusub.
- Xakamaynta tayada. Doorashooyinka injineernimada ee wanaagsan waxay yareeyaan shilalka la isku halleyn karo ee wax soo saarka.
Iska ilaali
- Hagaajinta hal bartilmaameed waxay qarin kartaa daciifnimada nidaamka ballaaran.
- Kaabayaasha dhaqaalaha iyo dayactirka inta badan waa la dhayalsadaa.
- Nabadgelyada iyo daldaloolada u fiirsashada ayaa kori kara marka nidaamyadu noqdaan kuwo aad u adag.
Tallaabooyinka xiga
- Qeex daahida, tayada, iyo bartilmaameedyada qiimaha ka hor inta aan la hirgelin.
- Benchmark marka la eego culeyska dhabta ah iyo xaaladaha xogta.
- La socodka qalabka khaladaadka, leexashada, iyo saamaynta isticmaalaha.
- U diyaari dib-u-noqoshada iyo dariiqyada jawaab-celinta dhacdada ka hor inta aanad miisaan.
Sii wad Sahaminta
Warqad bilaash ah
Hel warbixinta maalinlaha ah ee AI
Saddex sheeko oo la xaqiijiyay oo AI ah maalin kasta oo shaqo ah, oo lagu qoray Ingiriis fudud. Bilaash had iyo jeer, aan xayaysiis lahayn.
Hal iimayl maalin kasta oo shaqo ah. Ka bax hal gujin. Marna ma iibinno ama ma wadaagno ciwaankaaga.
Is tijaabi
Take the Gumbel-Softmax and Reparameterization quiz
Fikrad degdeg ah oo ku saabsan jawaab kasta, iyo shahaado la wadaajin karo oo leh aqoonsi la xaqiijin karo markaad ka gudubto koorso.
Taageer waxbarashada AI bilaashka ah. AI Understanding waa 501(c)(3) aan faa'iido doon ahayn — xayaysiis la'aan, albaab lacag ah ma jiro, had iyo jeer. Samee deeq
Su'aalaha soo noqnoqda
What is Gumbel-Softmax and Reparameterization?
Gumbel-Softmax waa khiyaamo u sahlaysa shabakadaha neural'ka 'muunad' qaybo kala duwan iyadoo weli lagu tababari karo faracyo hoose. Waa muhiim sababtoo ah faafinta dib-u-celinta caadi ahaan kuma socon karto doorasho aan toos ahayn, oo aan kala sooc lahayn.
Why is a categorical sample difficult to train through with ordinary backpropagation?
The guide explains gradients do not pass through a hard discrete jump.
What does the reparameterization trick aim to do?
The guide describes moving randomness into a noise source and preserving a smooth path.
How does Gumbel-Softmax approximate a categorical choice?
The guide gives the noisy softmax construction for differentiable categorical sampling.
What happens to the distribution as temperature becomes large?
High temperature yields a smoother distribution over categories.
What happens as temperature approaches zero?
The guide says low temperature sharpens toward a near one-hot vector.
Sii wad waxbarashada
Tilmaamaha la xidhiidha
Tilmaamayaal badan ayaa loo doortay mawduucan