InoteveraGaidhi rinotevera
Stochastic Gradient Descent ine Momentum
Basics
Basics GUIDE
Dropout uye Stochastic Regularization
Kudonhedza inzira yenguva dzose inodzima chidimbu chemaeuroni panguva yega yega nhanho yekudzidziswa, ichimanikidza network kuti ivake isina, inomiririra yakasimba.
Teerera kugwaro iriAI-yakarondedzerwa · ~2 min
Pa peji ino2 min verenga
Pfupiso
Yakave imwe yenzira dzakanyanya kupesvedzera dzekurwisa kuwandisa mukudzidza kwakadzama.
Kudzika Kwakadzika
Yakaunzwa neboka raHinton muna 2012, kusiya chikoro kunogadzirisa kusasimba kukuru kwemanetiweki makuru: neurons inogona kuchinjika, kudzidza kugadzirisa kukanganisa kweumwe neumwe nenzira dzinoshanda chete padhata rekudzidziswa. Pane yega yega yekupfuura panguva yekudzidziswa, kuregedza zvisina tsarukano kunogadzika kubuda kwe neuron yega yega kune zero neimwe mukana p (kazhinji 0.5 mumitsetse yakakora). Nekuti chero neuron inogona kunyangarika, network haigone kutsamira pahudyire husina kusimba uye inofanirwa kuparadzira ruzivo rwakakosha muzvikamu zvakawanda. Izvi zvinoita sekudzidzisa muunganidzwa wakakura wemataneti akatetepa anogovana uremu. Panguva yekuyedza kudonhedza kunodzimwa uye network izere inoshandiswa, ine activation yakayerwa kuitira kuti zvinotarisirwa kubuda zvienderane nekudzidziswa. Mhedzisiro yacho inowanzova nani generalization pamubhadharo wekudzidziswa kwenguva yakati rebei.
Kuitwa muNyika yeChokwadi
Kuwedzera Dropout layer ine p yakatenderedza 0.5 pakati pematanho akaomeswa emufananidzo kana zvinyorwa zvekirasi muPyTorch kana Keras.
Transformer modhi inoshandisa kudonhedza kune kutarisisa huremu uye kudyisa-mberi ma activation panguva yepretraining
Monte Carlo dropout, uko kudonha kunoramba kuri pakufungidzira kuburitsa fungidziro yekusavimbika yezvekurapa kana kuchengetedzeka-kwakakosha kufanotaura.
Stochastic kudzika (DropPath) zvisina tsarukano kusvetuka zvidhinha zvakasara kuti zvigadzirise zvakadzika network zvakaita seResNets uye echiono shandurudzo.
Ramangwana reKudonha uye Stochastic Regularization
Mune convolutional vision network, batch normalization yakabvisa zvakanyanya kudonha, asi misiyano inobudirira kumwe kunhu: ma transformer anoshandisa donhodzo kune kutarisisa uye nekudya-mberi maseru, uye DropPath (stochastic kudzika) inodonhedza zvese zvakasara zvidhinha. Monte Carlo dropout, iyo inoramba ichidonha ichishanda pakufungidzira, inoshandiswa kufungidzira kusava nechokwadi kwemuenzaniso. Tarisira kuti stochastic dhizaini irambe iri dhizaini inoshanduka, yakagadziridzwa padhizaini pane imwe chete yakagadziriswa resipi.
Kuzvishandisa
Nei zvichikosha
- Sarudzo dzakajeka. Inokubatsira kuparadzanisa zvakajeka zvichemo zvehunyanzvi kubva mumutauro wekushambadzira.
- Mutengo uye bhajeti. Iwe unogona kubvunza zvirinani kuita mibvunzo usati washandisa mari kana nguva.
- Team uye workflow. Zvikwata zvine nzwisiso yakagovaniswa inoita zvirinani chigadzirwa, mutemo, uye sarudzo dzekudzidza.
Chenjera
- Zvikwata zvakasiyana zvinogona kushandisa izwi rimwechete zvakasiyana, saka tsanangura nzvimbo nekukurumidza.
- Benchmarks inogona kutaridzika yakasimba nepo chaiyo-yenyika kuita isina kuenzana.
- Kuregeredza mhando yedata uye zvirongwa zvekuongorora zvinowanzogadzira mhedzisiro isina kusimba.
Matanho anotevera
- Tanga netsanangudzo yemutauro wakajeka yemhedzisiro yaunoda.
- Sarudza metric imwe yekubudirira uye imwe yekutadza mamiriro usati waedzwa.
- Mhanya mutyairi mudiki ane data remumiriri, kwete demo rakakwenenzverwa.
- Gwaro uko Dropout uye Stochastic Regularization inobatsira uye uko nzira dzakareruka dziri nani.
Ramba Uchiongorora
Tsamba yemahara
Tora iyo yemazuva ese AI pfupiso
Nhatu dzakasimbiswa dzeAI nyaya mangwanani ega ega pakati pevhiki, dzakanyorwa neChirungu chakajeka. Yemahara zvachose, hapana ads.
Imwe email zuva rega rega revhiki. Bvisa kunyoresa kamwe chete. Hatimbotengese kana kugovera kero yako.
Zviedze iwe pachako
Take the Dropout and Stochastic Regularization quiz
Mhinduro yepakarepo pamhinduro yega yega, uye chitupa chinogovaniswa chine ID inobatika kana wapedza kosi.
Tsigira dzidzo yemahara yeAI. AI Understanding ndeye 501(c)(3) isingabatsiri - hapana ads, hapana paywall, zvachose. Ita chipo
Mibvunzo inowanzo bvunzwa
Chii chinonzi Dropout uye Stochastic Regularization?
Kudonhedza inzira yenguva dzose inodzima chidimbu chemaeuroni panguva yega yega nhanho yekudzidziswa, ichimanikidza network kuti ivake isina, inomiririra yakasimba. Yakava imwe yenzira dzakanyanya kupesvedzera dzekurwisa kuwandisa mukudzidza kwakadzama.
Chii chinonzi dropout panguva yekudzidzira?
Kudonhedza zvisina tsarukano kunoisa zero imwe neimwe neuron ine mukana p panguva yekudzidziswa, saka yakasiyana yakatetepa sub-network inoshandiswa nhanho yega yega.
Sei kusiya chikoro kuchivandudza generalization?
Nekubvisa mayunitsi zvisina tsarukano, kudonhedza kunomisa neurons kuumba hukama husina kusimba hunongoshanda pane data rekudzidzisa, kuvandudza kusimba.
Chii chinoitika pakusiya pabvunzo (inference) nguva?
Pakunongedza network izere inomhanya nekudonha kwakadzimwa, uye activation inoyerwa saka zvinotarisirwa zvinobuda zvienderane nekugoverwa kwekudzidziswa.
Chiyero chekudonha chep = 0.5 muchikamu chinoreva kuti chii panguva yekudzidziswa?
Ne p = 0.5 neuron yega yega ine 50 muzana yemukana wekuve zero, saka paavhareji inenge hafu inodonhedzwa pane imwe yekupfuura.
Chii chinonzi kusiya chikoro chinowanzotsanangurwa sekufungidzira?
Sampling yakasarudzika sub-network nhanho imwe neimwe inofananidzira avhareji pamusoro pehuwandu hweexponential hweakagovaniswa-huremu network, chimiro chekuchipa ensembleng.
Ramba uchidzidza
Mazano anoenderana
Mamwe madhairekitori akasarudzirwa nyaya iyi