Normalisasioŋ ci lots
Normalisation batch xarala la buy yekketi eskaalu input yi ci bu nekk ci reso neuronal bi ci diiru tàggat, tax reso yu xóot yi di gëna gaaw di tàggat, gëna wóor.
Résumé
It became one of the most widely used tricks in deep learning.
Plongeur bu xóot
Bi done yi di jaar ci reso bu xóot bi, séddaleb valeur yi di dundal couche bu nekk dafay wéy di soppeeku ndax couche yu njëkk yi dañuy yeesal, te loolu dafay yeexal tàggat yaram. Normalisasioŋ lots, bi Ioffe ak Szegedy dugal ci 2015, dafay saafara jafe-jafe yii ci normalisee dugal bu nekk ci lots yu ndaw yi fi nekk, suko defee ñu am lu tollu ci zero moyenne ak variance unité. Ginaaw loolu mu jëfandikoo ñaari parametre yuñ mëna jàng, gamma ak beta, yuy may reso bi mu gëna yokk ba noppi mu toxal valeur yiñ normalisee su loolu amee njariñ, suko defee du ñàkk benn dooley representation. Payoff bi dafa yaatu: reso yi dañuy muñ njàng mu gëna rëy, dañuy daje ci jamono yu néew, ñoo gëna néew luñuy sensible ci initialisation poids, te dañuy faral di generalise tuuti. Li am solo mooy ni doxalin a ngi aju ci lim yiñ def ci lots yi, kon lots yu ndaw lool mën nañu ko indil jafe-jafe.
Gis-gis xarala
Ci màndarga bu nekk ci biir lote bu ndaw, norm lote bi dafay xayma moyenne ak variance lote bi, dindi moyenne bi, ba noppi xaaj ko ak deviation standard (ak benn epsilon bu ndaw ngir stabilite). Ginaaw loolu mu génne gamma yoon valeur normalisé boole ci beta, foofu lañuy jàngee gamma ak beta. Ci diiru tàggat-yaram dafay jëfandikoo lim ci lote yi ci noonu muy wéy di am moyenne yuy daw; ci waxtu inference dafay soppi ci moyenne yiñ denc, suko defee ñu baña aju ci yeneen misaal yi ñuy séddoo lote bi. Dañu koy faral di dugal ci diggante jéego ligneer bu benn couche ak fonction activation bi.
njeextalu pexe
Njëgg ak budget
Dogal yi architecture di jël dañuy indi njariñ ak njëgu liggéey bi ay at ci ginaaw.
dogal yu gëna leer
Njàngalem xarala yi dafay jàppale ekip yi ñu tànn li gën, te baña yam ci li gëna bees daal.
Xool kalite
Tanneef yu gëna baax ci wàllu ingeñër dina wàññi jafe-jafe yi ci wàllu wóor ci liggéey bi.
Ëlëgu normalisasioŋ ci lots
Normalisation ci lots mingi wéy di nekk fasu liggéey ci modelu vision convolutionnel, waaye limu aju ci lim ci lots lu jafe la ci reso yuy baaxoo, lots yu ndaw, ak tàggat yuñ séddale. Loolu moo waral ñu jël yeneen pexe yu melni normalisasioŋ couche, biy normalise ci man-man yi ci benn misaal te leegi mooy jiite architecture transformateur yi, boole ci normalisasioŋ groupe ak instance ci ay domen yuñ tànn. Gëstu dafay wéy ci reso yu amul normalisasioŋ yu méngoo ak njariñ yi ci initialisation ak scaling bu jaar yoon. Xaarandil ni normalisasioŋ dina wéy di am solo, ak xeetu tànneef biñ tànn ngir méngoo ak architecture bi.
Doxal ci àdduna dëgg
Dugalal ay couche norme ci biir benn classifier image ResNet suko defee mu mëna tàggat ak tolluwaayu jàng bu gëna kawe te dajaloo ci diir yu néew.
Dakkal tàggat reso convolutionnel bu xóot ngir nataali medsin yu njëkkoon wuute te amul normalisasioŋ.
Wàññi sensitivite ci tàmbali poid ci CNN buñ jagleel, suko defee ingénieur yi duñu yàgg di jëfandikoo loxo ngir tàmbali valeur yi.
Coppite ci lim ak xayma ci mode tàggat yaram dem ci moyenne yuñ denc sooy dugal benn model suko defee benn nataal buy wax luy waaja am mën nekk luy méngoo.
Risk yi ak balustrade yi
Optimize benn benchmark mën na nëbb ñakk kattan yu gëna yaatu ci sistem bi.
Njëg li ñuy fay ci infrastructure yi ak ci toppatoo dañuy faral di suufeel.
Bu sistem yi di gëna xawa jafee xam, jafe-jafe yi am ci wàllu kaaraange ak seetlu mën nañu gëna bari.
Roadmap ngir samp gi
Mandargal latency, kalite, ak njëg yi laata ngay jëfandikoo.
Benchmark ci biir sargal ak done yu dëggu.
Jumtukaay bi di saytu njuumte yi, derive bi ak njeextalu jëfandikukat bi.
Waajal rollback ak yooni tontu ci jafe-jafe yi laata ngay eskale.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Batch Normalization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
RMSNorm ak yamale bu njëkk
Laaj yi ñuy faral di laaj
What is Batch Normalization?
Normalisation batch xarala la buy yekketi eskaalu input yi ci bu nekk ci reso neuronal bi ci diiru tàggat, tax reso yu xóot yi di gëna gaaw di tàggat, gëna wóor. Nekk na benn ci pexe yiñ gëna jëfandikoo ci jàng bu xóot.
Luy normalisasioŋ ci lots?
Norme batch dafay yamale dugal yi ci benn layer ci jëfandikoo moyenne bi ak variance biñ xayma ci mini-batch bi fi nekk, di tëye activation yi ci rang bu dëgër bi tàggat bi di gëna dem.
Lan moo waral normalisasioŋ ci lots dafa am paramet gamma ak beta yuñ mëna jàng?
Ginaaw ñu normalisee ko ci zero moyenne ak variance unitaire, gamma ak beta dañuy bàyyi reso bi mu delloosi eskale bi ak soppaliwaat valeur yi su loolu amee njariñ, kon normalisasioŋ bi du tënk li layer bi mëna representé.
Lan mooy njariñ li ñuy gëna wax ci normalisasioŋ ci lots?
Suñu demee ba dugal ay layer ci eskaal bu baax, batch norm dafay may reso yi ñu tàggat seen bopp ci njàng mu gëna rëy, ñu gëna gaaw ci daje, ba noppi ñu gëna néew luñuy yëg ci initialisation.
Ci jamonoy inference (di wax luy am ci done yu bees), ban lim la normalisasioŋ batch di jëfandikoo?
Jëfandikoo lim ci wàllu lote ci inference dina tax ñu mëna wax luy am ci yeneen misaal yi bokk lote bi. Lu moy loolu, batch norm dafay jëfandikoo moyenne run fixe yuñ denc ci diiru tàggat yaram.
Lan moo waral normalisasioŋ lots mënu def lu baax ak lots yu tuuti lool?
Norme lote mingi aju ci xayma moyenne ak variance ci lote bi. Ak misaal yu néew, xayma yooyu dañu bari coow, te loolu mën na yàq tàggat yaram.