Tambali poid
Ni ngay defaree poid yi ñuy tàmbalee ci reso neuronal bi balaa ñuy tàmbali tàggat yaram, loolu mooy wane bu baax ndax siñaal yi ak gradient yi dina ñu wéy di wér ci diisaay yu xóot yi.
Résumé
Tambali bu baax mooy wuutale convergence bu gaaw ak model bu dul jàng.
Plongeur bu xóot
Laata ngay tàggat sa yaram, diisaay bu nekk dafa wara am valeur bu njëkk. Teg leen ñépp ci zero lu bon la: diisaay yu nuróo dañuy defar gradient yu nuróo, kon neuron yi duñu musa wuute - loolu mooy jafe-jafe biy yàq symétrie. Initialisation aleatoire dafay dog symétrie, waaye scale bi dafa am solo lool. Dafa yaatu lool, aktivasioŋ yi ak degrade yi dañuy kalaate; tuuti lool, ñuy réer. Xeetu njàngale yiñ tëral dañuy tànn variance bi lalu ci dayo couche bi ngir tëye variance siñaal bi ci diggante couche yi. Xavier (Glorot) dafay natt variance ci limu dugal ak genn yunit te méngoo ak reso tanh ak sigmoid. Moom (Kaiming) dafay tàmbalee ci limu dugal yi, ba noppi ReLU dafay sànni genn-wàllu dugal yi, moo tax mu nekk standard ci ReLU-based deep nets ak CNNs. Tambali bu baax dafay tëye tàggat yaram bu teel ba keroog normalisasioŋ ak optimisatër yiy méngoo jël loxo bi.
Gis-gis xarala
Luñu bëgga mooy nga fexe ba faraasu aktivasioŋ yi ak gradient yi duñu soppeeku ci couche bu nekk. Xavier dafa def variance poids ci 2 / (fan_in + fan_out), di ekilibre paas yi ci kanam ak ci ginaaw ngir mëna tàmbali symétrique. Dafay tàmbali jëfandikoo 2 / fan_in ndax ReLU dafay nul lu tollu ci genn-wàllu duggam, kon ñaari yoon variance bi dafay kompensaasioŋ siñaal bi ñàkk. Bias yi dañu leen di njëkka initialiser ci zero ndax symétrie bi dafa yàgg a dog ci poids yu bari yi.
njeextalu pexe
Njëgg ak budget
Dogal yi architecture di jël dañuy indi njariñ ak njëgu liggéey bi ay at ci ginaaw.
dogal yu gëna leer
Njàngalem xarala yi dafay jàppale ekip yi ñu tànn li gën, te baña yam ci li gëna bees daal.
Xool kalite
Tanneef yu gëna baax ci wàllu ingeñër dina wàññi jafe-jafe yi ci wàllu wóor ci liggéey bi.
Ëlëgu tàmbali poids
Normalisation layers ak connexions residuels taxna tàggat bi gëna néew luñu koy méngale ak initialisation bi, waaye mingi wéy di am solo ci reso yu xóot yi wala yu amul normalisation. Gëstu bu am solo dafay àndaale ak ay pexe yuñ méngale ak transformatër yi ak bàyyi xel, pexe yuy may reso yi ñu tàggat seen bopp te duñu am benn couche normalisation, ak theorie bu melni isométrie dynamique ak kernel tangent neuronal biy wax luy waaja tàggat ci ndoorte li kese. Initialisation bu aju ci done, mooy kalibre balance yi ci benn misaal, beneen yoon la buy màgg.
Doxal ci àdduna dëgg
CNN biy jëfandikoo ReLU dafay tàmbali ak He initialisation suko defee stacks convolutionnel yu xóot yi di tàggat te duñu réer siñaal yi.
Reseau bu am aktivasioŋ tanh dafay jëfandikoo initialisation Xavier ngir tëye variance aktivasioŋ bi ci diisaay yépp.
Ab ingénieur bu initialiser poids yépp ci zero ci anam wu jaarul yoon, dafay gis ni reso bi mënul jàng ndax neuron bu nekk dafay wéy di nekk benn.
Default yi ci kaadar bi (Kaiming bu PyTorch, uniforme Glorot bu Keras) dañuy jëfandikoo ndoortelu njàngale ci saasi suñu sosee benn couche.
Risk yi ak balustrade yi
Optimize benn benchmark mën na nëbb ñakk kattan yu gëna yaatu ci sistem bi.
Njëg li ñuy fay ci infrastructure yi ak ci toppatoo dañuy faral di suufeel.
Bu sistem yi di gëna xawa jafee xam, jafe-jafe yi am ci wàllu kaaraange ak seetlu mën nañu gëna bari.
Roadmap ngir samp gi
Mandargal latency, kalite, ak njëg yi laata ngay jëfandikoo.
Benchmark ci biir sargal ak done yu dëggu.
Jumtukaay bi di saytu njuumte yi, derive bi ak njeextalu jëfandikukat bi.
Waajal rollback ak yooni tontu ci jafe-jafe yi laata ngay eskale.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Weight Initialization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Stochastic moyenne poids
Laaj yi ñuy faral di laaj
Luy tàmbali poid?
Ni ngay defaree poid yi ñuy tàmbalee ci reso neuronal bi balaa ñuy tàmbali tàggat yaram, loolu mooy wane bu baax ndax siñaal yi ak gradient yi dina ñu wéy di wér ci diisaay yu xóot yi. Tambali bu baax mooy wuutale convergence bu gaaw ak model bu dul jàng.
Lan moo waral initialiser poids yépp ci zero njuumte bu rëy la?
Ak diisaay yu nuróo, neuron bu nekk dafay xayma benn génne ak benn degrade, suko defee ñuy yeesal bu nuróo te couche bi du musa jàng màndarga yu wuute.
Mu (Kaiming) initialisasioŋ bi ñu defar ngir ban fonction aktivasioŋ?
Dafay tàmbali eskaalu variance ci 2 / fan_in ngir kompensaasioŋ ReLU buy nul lu tollu ci genn-wàll ci ay dugalam.
Lan mooy mébet bu mag bi ci pexe tàmbali poid bu lalu ci yoon?
Xeetu xew-xew yu melni Xavier ak He dañuy tànn variance poids ci dayo layer yi suko defee siñaal yi duñu ni mes wala ñuy kalaate suñuy tasaaroo.
Xavier (Glorot) tàmbali bi gëna baax ci reso yi jëfandikoo ban tàmbali?
Xavier dafay ekilibre variance ci kanam ak ci ginaaw ngir am symétrie, saturasioŋ yu melni tanh ak sigmoid.
Lu tax mu tàmbali jëfandikoo variance 2 / fan_in moo gën 1 / fan_in?
ReLU du joxe lenn ludul duggal yu baax, sànni lu tollu ci genn-wàllu siñaal bi, kon yokk ñaari yoon variance bi dafay delloosi variance activation biñ doon seentu.