Fara nauyi
Yadda kuke saita ma'aunin farawa na cibiyar sadarwa na jijiyoyi kafin a fara horo, wanda ke da ƙarfi sosai ko sigina da gradients suna da lafiya ta cikin yadudduka masu zurfi.
Dubawa
Good initialization is the difference between fast convergence and a model that never learns.
Zurfafa nutsewa
Kafin horo, kowane nauyi yana buƙatar ƙimar farawa. Sanya su duka zuwa sifili yana da mutuwa: ma'aunin nauyi iri ɗaya yana haifar da gradients iri ɗaya, don haka neurons ba su taɓa bambanta ba - wannan ita ce matsalar warwarewar ƙima. Farawa bazuwar yana karya daidaito, amma ma'aunin yana da mahimmanci. Yayi girma da yawa da kunnawa da gradients suna fashewa; ƙanƙanta kuma sun ɓace. Tsare-tsare masu ƙa'ida sun zaɓi bambance-bambancen dangane da girman Layer don kiyaye bambancin sigina daidai gwargwado a tsakanin yadudduka. Xavier (Glorot) farkon farawa yana daidaita bambance-bambance ta adadin shigarwar da raka'o'in fitarwa kuma ya dace da cibiyoyin sadarwa na tanh da sigmoid. Ya (Kaiming) ƙaddamar da ma'auni ta adadin abubuwan da aka shigar da asusun don ReLU ya watsar da rabin abubuwan da aka shigar, yana mai da shi ma'auni na tushen ReLU mai zurfi da CNNs. Kyakkyawan farawa yana kiyaye horon da wuri har sai an daidaita da inganta haɓakawa.
Fahimtar Fasaha
Manufar ita ce kiyaye bambance-bambancen kunnawa da gradients akai-akai daga Layer zuwa Layer. Xavier yana saita bambance-bambancen nauyi zuwa 2 / (fan_in + fan_out), yana daidaita wucewar gaba da baya don kunna madaidaici. Yana farawa yana amfani da 2 / fan_in saboda ReLU yana fitar da kusan rabin abubuwan shigarsa, don haka ninka bambancin yana rama wannan siginar da ta ɓace. Ana fara nuna son zuciya zuwa sifili tun da an riga an karye sifili ta ma'aunin bazuwar.
Dabarun Tasiri
Kudin da kasafin kuɗi
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Shawarwari masu haske
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Kula da inganci
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Makomar Farko Nauyi
Daidaita yadudduka da ragowar haɗin kai sun sanya horarwa ba ta da hankali ga ainihin farawa, amma har yanzu yana da mahimmanci ga cibiyoyin sadarwa masu zurfi ko daidaitawa. Bincike mai aiki ya haɗa da tsare-tsare waɗanda aka keɓance da masu taswira da hankali, hanyoyin da ke barin cibiyoyin sadarwa su yi horo ba tare da wani nau'i na daidaitawa ba, da ka'idar kamar isometry mai ƙarfi da kernel tangent na jijiya wanda ke hasashen iyawar horo daga farawa kaɗai. Ƙaddamar da tushen bayanai, wanda ke daidaita ma'auni daga tsarin samfurin, wata hanya ce mai girma.
Aiwatar da Gaskiyar Duniya
CNN da ke amfani da ayyukan ReLU an fara farawa tare da Ƙaddamarwa don haka zurfin juzu'i na jujjuyawar jirgin ƙasa ba tare da ɓata sigina ba.
Cibiyar sadarwa tare da kunnawa tanh tana amfani da ƙaddamarwar Xavier don kiyaye bambancin kunnawa a tsakanin yadudduka.
Injiniya wanda ya fara ƙaddamar da duk ma'aunin nauyi zuwa sifili ba da gangan ba yana ganin hanyar sadarwa ta kasa koyo saboda kowane neuron yana zama iri ɗaya.
Matsalolin Tsari (PyTorch's Kaiming, Keras's Glorot uniform) suna amfani da ƙa'idar farawa ta atomatik lokacin da aka ƙirƙiri Layer.
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Weight Initialization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Matsakaicin Nauyin Stochastic
Tambayoyin da ake yawan yi
What is Weight Initialization?
Yadda kuke saita ma'aunin farawa na cibiyar sadarwa na jijiyoyi kafin a fara horo, wanda ke da ƙarfi sosai ko sigina da gradients suna da lafiya ta cikin yadudduka masu zurfi. Kyakkyawan farawa shine bambanci tsakanin haɗuwa da sauri da samfurin da bai taɓa koyo ba.
Me yasa fara duk ma'aunin nauyi zuwa sifili kuskure ne mai mutuwa?
Tare da ma'auni iri ɗaya, kowane neuron yana ƙididdige fitarwa iri ɗaya da gradient, don haka suna ɗaukaka iri ɗaya kuma Layer ba zai taɓa koyon fasali na musamman ba.
Shi (Kaiming) farawa an tsara shi musamman don wane aikin kunnawa?
Ya ƙaddamar da ma'auni bambance-bambance ta 2 / fan_in don ramawa ReLU sifilin kusan rabin abubuwan shigarta.
Menene babban burin tsare-tsaren fara nauyi mai ka'ida?
Tsare-tsare kamar Xavier kuma Ya zaɓi bambance-bambancen nauyi daga masu girma dabam don haka sigina ba ya ɓace ko fashe yayin da suke yaduwa.
Farkon Xavier (Glorot) ya fi dacewa da cibiyoyin sadarwa ta amfani da waɗanne kunnawa?
Xavier yana daidaita bambance-bambancen gaba da baya don daidaitawa, aiki mai gamsarwa kamar tanh da sigmoid.
Me yasa farawa yana amfani da bambance-bambancen 2 / fan_in maimakon 1 / fan_in?
ReLU yana wucewa kawai ingantattun bayanai, yana watsar da kusan rabin siginar, don haka ninka bambance-bambancen yana dawo da saɓanin kunnawa da ake tsammanin.