InoteveraGaidhi rinotevera
Stochastic Weight Avhareji
Tekinoroji
Nhungamiro yehunyanzvi
Weight Initialization
Maseti aunoita neural network yekutanga huremu isati yatanga kudzidziswa, iyo inoumba zvakasimba kana masaini uye gradients zvinogara zvine hutano kuburikidza neakadzama layers.
Teerera kugwaro iriAI-yakarondedzerwa · ~2 min
Pa peji ino2 min verenga
Pfupiso
Kutanga kwakanaka ndiko mutsauko pakati pekukurumidza kusangana uye modhi isingambodzidzi.
Kudzika Kwakadzika
Usati wadzidzira, uremu hwese hunoda kukosha kwekutanga. Kudziisa ku zero kunouraya: uremu hwakafanana hunoburitsa magradients akafanana, saka neuroni haambosiyanise - iri ndiro dambudziko rekutyora symmetry. Random kutanga kunotyora symmetry, asi chiyero chakakosha zvikuru. Yakakura kwazvo uye activation uye gradients inoputika; diki uye dzinonyangarika. Zvirongwa zvakatemerwa zvinosarudza musiyano unoenderana nehukuru hwesaizi kuti uchengetedze mutsauko wechiratidzo unoramba uripo pamatanho. Xavier (Glorot) kutanga kuyera kusiyanisa nenhamba yekupinza pamwe neinobuda mayuniti uye inokodzera tanh uye sigmoid network. Iye (Kaiming) ekutanga anoyera nenhamba yezvipimo uye maakaundi eReLU achirasa hafu yezvipo zvayo, zvichiita kuti ive chiyero cheReLU-based mambure akadzika uye CNNs. Kutanga kwakanaka kunochengeta kudzidziswa kwekutanga kwakagadzikana kusvika kujaira uye adaptive optimizers zvatora.
Kuitwa muNyika yeChokwadi
A CNN inoshandisa ReLU activation inotangwa neIye kutanga saka yakadzika convolutional stacks chitima pasina kunyangarika masaini.
Netiweki ine tanh activation inoshandisa Xavier kutanga kuchengetedza activation musiyano yakagadzikana pamatanho.
Injiniya anotanga netsaona zviremu zvese kusvika zero anoona network ichitadza kudzidza nekuti neuron yega yega inogara yakafanana.
Framework defaults (PyTorch's Kaiming, Keras's Glorot yunifomu) shandisa misimboti yekutanga otomatiki kana layer yagadzirwa.
Ramangwana reHuremu Kutanga
Normalization layers uye zvakasara zvinongedzo zvakaita kuti kudzidziswa kuite kushomeka kune chaiyo yekutanga, asi ichiri kukosha kune yakadzika kwazvo kana normalization-yemahara network. Tsvagiridzo yakasimba inosanganisira zvirongwa zvakanangana nevashanduri uye kutarisisa, nzira dzinoita kuti network dzidzidzire pasina zvimiro zvenguva dzose, uye dzidziso senge dynamical isometry uye neural tangent kernel inofanotaura kudzidziswa kubva pakutanga chete. Dhata-inotsamira kutanga, iyo inogadzirisa zviyero kubva kumuenzaniso batch, ndeimwe nzira inokura.
Kuzvishandisa
Nei zvichikosha
- Mutengo uye bhajeti. Zvisarudzo zvezvivakwa zvinotyaira kuita uye mutengo wekushandisa kwemakore.
- Sarudzo dzakajeka. Dzidzo yehunyanzvi inobatsira zvikwata kusarudza murwi wakakodzera, kwete iwo mutsva chete.
- Kudzora kwemhando yepamusoro. Sarudzo dzeinjiniya dziri nani dzinoderedza zviitiko zvekuvimbika mukugadzira.
Chenjera
- Kugadzirisa imwe bhenji kunogona kuvanza yakafara system kushaya simba.
- Infrastructure uye mari yekugadzirisa inowanzotarisirwa pasi.
- Chengetedzo uye kucherechedzwa mapundu anogona kukura sezvo masisitimu anowedzera kuoma.
Matanho anotevera
- Tsanangura latency, mhando, uye mutengo zvinangwa usati waitwa.
- Benchmark pasi pechokwadi mutoro uye data mamiriro.
- Chishandiso chekutarisa zvikanganiso, kudonha, uye mushandisi maitiro.
- Gadzirira nzira dzekudzosera kumashure uye dzezviitiko usati wawedzera.
Ramba Uchiongorora
Tsamba yemahara
Tora iyo yemazuva ese AI pfupiso
Nhatu dzakasimbiswa dzeAI nyaya mangwanani ega ega pakati pevhiki, dzakanyorwa neChirungu chakajeka. Yemahara zvachose, hapana ads.
Imwe email zuva rega rega revhiki. Bvisa kunyoresa kamwe chete. Hatimbotengese kana kugovera kero yako.
Zviedze iwe pachako
Take the Weight Initialization quiz
Mhinduro yepakarepo pamhinduro yega yega, uye chitupa chinogovaniswa chine ID inobatika kana wapedza kosi.
Tsigira dzidzo yemahara yeAI. AI Understanding ndeye 501(c)(3) isingabatsiri - hapana ads, hapana paywall, zvachose. Ita chipo
Mibvunzo inowanzo bvunzwa
Chii chinonzi Weight Initialization?
Maseti aunoita neural network yekutanga huremu isati yatanga kudzidziswa, iyo inoumba zvakasimba kana masaini uye gradients zvinogara zvine hutano kuburikidza neakadzama layers. Kutanga kwakanaka ndiko mutsauko pakati pekukurumidza kusangana uye modhi isingambodzidzi.
Sei kutanga zviremu zvese kusvika zero iri chikanganiso chinouraya?
Nehuremu hwakafanana, neuron yega yega inokokorodza kuburitsa kwakafanana uye gradient, saka ivo vanogadziridza zvakafanana uye iyo layer haigone kudzidza akasiyana maficha.
Iye (Kaiming) kutanga kwakanyatso kugadzirirwa iyo activation basa?
Anotanga anoyera kusiyanisa ne2 / fan_in kutsiva ReLU kuburitsa inenge hafu yezvipo zvayo.
Ndechipi chinangwa chikuru chezvirongwa zvekutanga zvehuremu?
Zvirongwa zvakaita seXavier uye Iye anosarudza huremu mutsauko kubva kune akaturikidzana saizi kuitira kuti masaini asapera kana kuputika sezvaanoparadzira.
Xavier (Glorot) kutanga kwakanyatsokodzera kunetiweki uchishandisa kupi activation?
Xavier anoyera kumberi nekumashure musiyano we symmetric, saturating activation senge tanh uye sigmoid.
Sei Iye kutanga achishandisa musiyano we 2 / fan_in pane 1 / fan_in?
ReLU inongopfuura zvinopinda zvakanaka, ichirasa inenge hafu yechiratidzo, saka kupeta kaviri mutsauko kunodzoreredza inotarisirwa activation musiyano.
Ramba uchidzidza
Mazano anoenderana
Mamwe madhairekitori akasarudzirwa nyaya iyi