Dropout uye Stochastic Regularization
Kudonhedza inzira yenguva dzose inodzima chidimbu chemaeuroni panguva yega yega nhanho yekudzidziswa, ichimanikidza network kuti ivake isina, inomiririra yakasimba.
Pfupiso
It became one of the most influential techniques for fighting overfitting in deep learning.
Kudzika Kwakadzika
Yakaunzwa neboka raHinton muna 2012, kusiya chikoro kunogadzirisa kusasimba kukuru kwemanetiweki makuru: neurons inogona kuchinjika, kudzidza kugadzirisa kukanganisa kweumwe neumwe nenzira dzinoshanda chete padhata rekudzidziswa. Pane yega yega yekupfuura panguva yekudzidziswa, kuregedza zvisina tsarukano kunogadzika kubuda kwe neuron yega yega kune zero neimwe mukana p (kazhinji 0.5 mumitsetse yakakora). Nekuti chero neuron inogona kunyangarika, network haigone kutsamira pahudyire husina kusimba uye inofanirwa kuparadzira ruzivo rwakakosha muzvikamu zvakawanda. Izvi zvinoita sekudzidzisa muunganidzwa wakakura wemataneti akatetepa anogovana uremu. Panguva yekuyedza kudonhedza kunodzimwa uye network izere inoshandiswa, ine activation yakayerwa kuitira kuti zvinotarisirwa kubuda zvienderane nekudzidziswa. Mhedzisiro yacho inowanzova nani generalization pamubhadharo wekudzidziswa kwenguva yakati rebei.
Technical Insight
Panguva yekudzidziswa imwe neimwe unit inochengetwa ine mukana (1 minus p) kuburikidza neasina bhinari mask, saka akasiyana ma-network anotorwa batch yega yega. Mafuremu echizvino zvino anoshandisa invertout dropout: ma activation akasara akakamurwa ne (1 minus p) panguva yechitima, saka hapana kuyera kunodiwa pakunongedza. Kusarongeka uku kunopinza ruzha runoodza moyo kuchinjika uye kufungidzira avhareji pamusoro pehuwandu hwehuwandu hweakagovaniswa-huremu sub-network, yakachipa nzira yekuunganidza.
Strategic Impact
Sarudzo dzakajeka
Inokubatsira kuparadzanisa zvakajeka zvichemo zvehunyanzvi kubva mumutauro wekushambadzira.
Mutengo uye bhajeti
Iwe unogona kubvunza zvirinani kuita mibvunzo usati washandisa mari kana nguva.
Team uye workflow
Zvikwata zvine nzwisiso yakagovaniswa inoita zvirinani chigadzirwa, mutemo, uye sarudzo dzekudzidza.
Ramangwana reKudonha uye Stochastic Regularization
Mune convolutional vision network, batch normalization yakabvisa zvakanyanya kudonha, asi misiyano inobudirira kumwe kunhu: ma transformer anoshandisa donhodzo kune kutarisisa uye nekudya-mberi maseru, uye DropPath (stochastic kudzika) inodonhedza zvese zvakasara zvidhinha. Monte Carlo dropout, iyo inoramba ichidonha ichishanda pakufungidzira, inoshandiswa kufungidzira kusava nechokwadi kwemuenzaniso. Tarisira kuti stochastic dhizaini irambe iri dhizaini inoshanduka, yakagadziridzwa padhizaini pane imwe chete yakagadziriswa resipi.
Real-World Implementation
Kuwedzera Dropout layer ine p yakatenderedza 0.5 pakati pematanho akaomeswa emufananidzo kana zvinyorwa zvekirasi muPyTorch kana Keras.
Transformer modhi inoshandisa kudonhedza kune kutarisisa huremu uye kudyisa-mberi ma activation panguva yepretraining
Monte Carlo dropout, uko kudonha kunoramba kuri pakufungidzira kuburitsa fungidziro yekusavimbika yezvekurapa kana kuchengetedzeka-kwakakosha kufanotaura.
Stochastic kudzika (DropPath) zvisina tsarukano kusvetuka zvidhinha zvakasara kuti zvigadzirise zvakadzika network zvakaita seResNets uye echiono shandurudzo.
Njodzi & Guardrails
Zvikwata zvakasiyana zvinogona kushandisa izwi rimwechete zvakasiyana, saka tsanangura nzvimbo nekukurumidza.
Benchmarks inogona kutaridzika yakasimba nepo chaiyo-yenyika kuita isina kuenzana.
Kuregeredza mhando yedata uye zvirongwa zvekuongorora zvinowanzogadzira mhedzisiro isina kusimba.
Implementation Roadmap
Tanga netsanangudzo yemutauro wakajeka yemhedzisiro yaunoda.
Sarudza metric imwe yekubudirira uye imwe yekutadza mamiriro usati waedzwa.
Mhanya mutyairi mudiki ane data remumiriri, kwete demo rakakwenenzverwa.
Gwaro uko Dropout uye Stochastic Regularization inobatsira uye uko nzira dzakareruka dziri nani.
Ramba Uchiongorora
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Dropout and Stochastic Regularization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gaidhi rinotevera
Stochastic Gradient Descent ine Momentum
Mibvunzo inowanzo bvunzwa
What is Dropout and Stochastic Regularization?
Kudonhedza inzira yenguva dzose inodzima chidimbu chemaeuroni panguva yega yega nhanho yekudzidziswa, ichimanikidza network kuti ivake isina, inomiririra yakasimba. Yakava imwe yenzira dzakanyanya kupesvedzera dzekurwisa kuwandisa mukudzidza kwakadzama.
Chii chinonzi dropout panguva yekudzidzira?
Kudonhedza zvisina tsarukano kunoisa zero imwe neimwe neuron ine mukana p panguva yekudzidziswa, saka yakasiyana yakatetepa sub-network inoshandiswa nhanho yega yega.
Sei kusiya chikoro kuchivandudza generalization?
Nekubvisa mayunitsi zvisina tsarukano, kudonhedza kunomisa neurons kuumba hukama husina kusimba hunongoshanda pane data rekudzidzisa, kuvandudza kusimba.
Chii chinoitika pakusiya pabvunzo (inference) nguva?
Pakunongedza network izere inomhanya nekudonha kwakadzimwa, uye activation inoyerwa saka zvinotarisirwa zvinobuda zvienderane nekugoverwa kwekudzidziswa.
Chiyero chekudonha chep = 0.5 muchikamu chinoreva kuti chii panguva yekudzidziswa?
Ne p = 0.5 neuron yega yega ine 50 muzana yemukana wekuve zero, saka paavhareji inenge hafu inodonhedzwa pane imwe yekupfuura.
Chii chinonzi kusiya chikoro chinowanzotsanangurwa sekufungidzira?
Sampling yakasarudzika sub-network nhanho imwe neimwe inofananidzira avhareji pamusoro pehuwandu hweexponential hweakagovaniswa-huremu network, chimiro chekuchipa ensembleng.