Ukuyeka kanye ne-Stochastic Regularization
Ukukhipha i-dropout iqhinga lokujwayela elivala ngokungahleliwe ingxenye yama-neurons phakathi nesinyathelo ngasinye sokuqeqesha, okuphoqa inethiwekhi ukuthi yakhe izethulo ezingafuneki, eziqinile.
Uhlolojikelele
It became one of the most influential techniques for fighting overfitting in deep learning.
I-Deep Dive
Okwethulwa iqembu lika-Hinton cishe ngo-2012, ukuyeka ukufunda kubhekana nobuthakathaka obuyinhloko bamanethiwekhi amakhulu: ama-neurons angakwazi ukuzivumelanisa nezimo, afunde ukulungisa amaphutha womunye nomunye ngezindlela ezisebenza kuphela kudatha yokuqeqeshwa. Kukho konke ukudlula okuya phambili phakathi nokuqeqeshwa, ukuyeka ukuyeka ngokungahleliwe kusetha okukhiphayo kwe-neuron ngayinye kuqanda ngamathuba athile okuthi p (ngokuvamile angu-0.5 ezendlalelo eziminyene). Ngenxa yokuthi noma iyiphi i-neuron ingase inyamalale, inethiwekhi ayikwazi ukuncika ebudlelwaneni obubuthakathaka futhi kufanele isabalalise ulwazi oluwusizo kuwo wonke amayunithi. Lokhu kufana nokuqeqesha iqoqo elikhulu lamanethiwekhi amancane abelana ngezisindo. Ngesikhathi sokuhlola ukuyeka kuyavalwa futhi kusetshenziswe inethiwekhi egcwele, nokwenziwa kusebenze kukalwe ukuze okuphumayo okulindelekile kufane nokuqeqeshwa. Umphumela uba ukujwayela okungcono kakhulu ngezindleko zokuqeqeshwa okude kancane.
I-Technical Insight
Phakathi nokuqeqeshwa iyunithi ngayinye igcinwa inethuba (1 khipha p) kusetshenziswa imaski kanambambili engahleliwe, ngakho-ke amanethiwekhi amancane ahlukene athathwa isampula ngayinye. Izinhlaka zesimanje zisebenzisa ukuyeka isikolo okuhlanekezelwe: ukwenza kusebenze okusindile kuhlukaniswa ngo-(1 minus p) ngesikhathi sesitimela, ngakho-ke asikho isikali esidingekayo lapho kucatshangwa khona. Lokhu kungenzeki kungenisa umsindo ovimbela ukuzivumelanisa nezimo futhi kulinganisele isilinganiso esingaphezu kwenombolo ye-exponential yamanethiwekhi angaphansi kwesisindo esabelwe, indlela eshibhile yokuhlanganisa.
I-Strategic Impact
Izinqumo ezicacile
Kukusiza ukuthi uhlukanise izimangalo ezicacile zobuchwepheshe kusukela olimini lokumaketha.
Izindleko kanye nesabelomali
Ungabuza imibuzo yokusebenzisa kangcono ngaphambi kokusebenzisa imali noma isikhathi.
Ithimba kanye nokusebenza komsebenzi
Amaqembu anokuqonda okwabiwe enza izinqumo ezingcono zomkhiqizo, inqubomgomo, nokufunda.
Ikusasa Lokuyeka kanye Ne-Stochastic Regularization
Kumanethiwekhi ombono we-convolutional, i-batch normalization iye yashiya kakhulu ukuyeka okujwayelekile, kodwa okuhlukile kuyachuma kwenye indawo: ama-transformer afaka i-dropout ekunakekelweni nasekudluliseleni phambili izendlalelo, kanti i-DropPath (ukujula kwesitoko) yehlisa amabhlogo ayinsalela. I-Monte Carlo eyehlayo, egcina ukuyeka isikole isebenza ekucabangeni, isetshenziselwa ukulinganisa ukungaqiniseki kwemodeli. Lindela ukumiswa kwe-stochastic ukuze kuhlale kuyikhithi yamathuluzi eguquguqukayo, eguqulelwe kwisakhiwo ngasinye kuneresiphi eyodwa engashintshi.
Ukuqaliswa Komhlaba Wangempela
Ukwengeza isendlalelo se-Dropout esino-p cishe u-0.5 phakathi kwezingqimba eziminyene zesithombe noma isihlukanisi sombhalo ku-PyTorch noma i-Keras
Amamodeli e-Transformer asebenzisa ukuyeka ukuyeka ezisindweni zokunaka kanye nokwenza kusebenze okudlulela phambili phakathi nokuqeqeshwa kwangaphambili
Ukuyeka i-Monte Carlo, lapho ukuyeka ukufunda kuhlale kucatshangelwa ukukhiqiza izilinganiso zokungaqiniseki zokubikezela okubalulekile kwezokwelapha noma ukuphepha
Ukujula kwe-Stochastic (i-DropPath) kweqa ngokungahleliwe amabhulokhi ayinsalela ukuze kwenziwe amanethiwekhi ajule kakhulu njenge-ResNets neziguquli zombono
Izingozi & Guardrails
Amaqembu ahlukene angasebenzisa igama elifanayo ngokuhlukile, ngakho chaza ububanzi kusenesikhathi.
Amabhentshimakhi angabukeka eqinile kuyilapho ukusebenza komhlaba wangempela kungalingani.
Ukuziba ikhwalithi yedatha nezinhlelo zokuhlaziya kuvame ukudala imiphumela entekenteke.
Ukuqalisa Umhlahlandlela
Qala ngencazelo yolimi olulula yomphumela oyidingayo.
Khetha imethrikhi eyodwa yempumelelo nesimo esisodwa sokuhluleka ngaphambi kokuhlolwa.
Qalisa umshayeli omncane onedatha emele, hhayi isethi yedemo ephucuziwe.
Idokhumenti lapho i-Dropout kanye ne-Stochastic Regularization kusiza nalapho izindlela ezilula zingcono.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Dropout and Stochastic Regularization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Umhlahlandlela olandelayo
Ukwehla kwe-Stochastic Gradient nge-Momentum
Imibuzo evame ukubuzwa
What is Dropout and Stochastic Regularization?
Ukukhipha i-dropout iqhinga lokujwayela elivala ngokungahleliwe ingxenye yama-neurons phakathi nesinyathelo ngasinye sokuqeqesha, okuphoqa inethiwekhi ukuthi yakhe izethulo ezingafuneki, eziqinile. Kube ngenye yezindlela ezinomthelela omkhulu ekulweni nokugcwala ngokweqile ekufundeni okujulile.
Kwenzani ukuyeka esikoleni ngesikhathi sokuqeqeshwa?
Ukuyeka ukuyeka ngokungahleliwe kwenza i-neuron ngayinye ibe ngu-zero ngamathuba okuthi p phakathi nokuqeqeshwa, ngakho-ke kusetshenziswa inethiwekhi encane ehlukile ehlukile isinyathelo ngasinye.
Kungani ukuyeka ukufunda kuthuthukisa ukujwayelekile?
Ngokususa amayunithi ngokungahleliwe, ukuyeka ukuyeka kumisa ama-neurons ekwenzeni ubambiswano olubuthakathaka olusebenza kuphela kudatha yokuqeqeshwa, ukuthuthukisa ukuqina.
Kwenzekani ngokuyeka esikoleni ngesikhathi sokuhlolwa (okucatshangwayo)?
Uma kucatshangelwa ukuthi inethiwekhi egcwele iyasebenza futhi ukuyeka ukuphuma kukhutshaziwe, futhi ukwenza kusebenze kuyalinganiswa ukuze imiphumela elindelekile ifane nokusabalalisa kokuqeqeshwa.
Izinga lokuyeka phakathi kuka-p = 0.5 ongqimbeni lisho ukuthini phakathi nokuqeqeshwa?
Nge-p = 0.5 i-neuron ngayinye inethuba elingamaphesenti angu-50 lokuba i-zeroed, ngakho-ke ngokwesilinganiso cishe uhhafu wehliswa ngokudlula okuya phambili.
Yini evame ukuchazwa ngokuthi iwukulinganisa?
Ukusampula inethiwekhi engaphansi ehlukile isinyathelo ngasinye silinganisa isilinganiso esingaphezu kwenombolo ecacile yamanethiwekhi anesisindo esabelwe, uhlobo lokuhlanganisa okushibhile.