Mmalite ibu
Otu esi edobe nha mmalite nke netwọkụ akwara tupu ọzụzụ amalite, nke na-akpụzi ike ma akara na gradients na-adị mma site n'ígwé dị omimi.
Nchịkọta
Good initialization is the difference between fast convergence and a model that never learns.
Ime miri emi
Tupu ọzụzụ, arọ ọ bụla chọrọ uru mmalite. Itinye ha niile ka ọ bụrụ efu na-egbu egbu: nha nha na-emepụta gradients yiri nke ahụ, yabụ neuron adịghị ekewa - nke a bụ nsogbu na-emebi symmetry. Mmalite nke enweghị usoro na-agbaji symmetry, mana ọnụ ọgụgụ ahụ dị oke mkpa. Oke buru ibu na activation na gradients na-agbawa; dị obere ma ha na-apụ n'anya. Atụmatụ ndị nwere ụkpụrụ na-ahọrọ mgbanwe dị iche iche dabere na nha oyi akwa iji mee ka mgbanwe mgbaama na-adịgide adịgide n'ofe ọkwa. Xavier (Glorot) mmalite mmalite na-atụba ọdịiche site na ọnụọgụ ntinye gbakwunyere nkeji mmepụta yana dabara tanh na netwọk sigmoid. Ọ (Kaiming) mmalite mmalite site na ọnụ ọgụgụ ntinye na akaụntụ maka ReLU na-atụfu ọkara ntinye ya, na-eme ka ọ bụrụ ọkọlọtọ maka ụgbụ miri emi dabeere na ReLU na CNN. Mma mmalite dị mma na-edobe ọzụzụ n'oge ka ọ kwụsie ike ruo mgbe nhazi nke ọma na ndị na-eme mgbanwe ga-eweghara.
Nghọta nka nka
Ebumnobi bụ idobe ọdịiche nke mmemme na gradients mgbe niile site na oyi akwa ruo oyi akwa. Xavier na-edobe ọdịiche dị arọ na 2 / (fan_in + fan_out), na-edozi ngafe na-aga n'ihu na azụ azụ maka mmegharị symmetric. Mmalite ya na-eji 2 / fan_in n'ihi na ReLU na-ewepụ ihe dị ka ọkara ntinye ya, yabụ na-agbanwe mgbanwe ahụ na-akwụ ụgwọ maka mgbama ahụ furu efu. A na-ebutekarị echiche ọjọọ ka ọ bụrụ efu ebe ọ bụ na ejirila ihe dị arọ emebiela symmetry.
Mmetụta atụmatụ
Ọnụ ego na mmefu ego
Mkpebi ihe owuwu ụlọ na-akwalite arụmọrụ yana ọnụ ahịa ọrụ ruo ọtụtụ afọ.
Mkpebi doro anya
Nkà mmụta nka na-enyere ndị otu egwuregwu aka ịhọrọ nchịkọta ziri ezi, ọ bụghị naanị nke kachasị ọhụrụ.
Quality akara
Nhọrọ injinia ka mma na-ebelata ihe omume ntụkwasị obi na mmepụta.
Ọdịnihu nke mmalite ibu ibu
Nhazi nhazi na njikọ ndị fọdụrụ emeela ka ọzụzụ na-adịchaghị mma maka mmalite mmalite, mana ọ ka dị mkpa maka netwọkụ dị omimi ma ọ bụ nke na-enweghị nhazi. Nchọpụta na-arụsi ọrụ ike gụnyere atụmatụ ahaziri maka mgbanwe na nlebara anya, ụzọ ndị na-ahapụ netwọkụ ka ha na-azụ na-enweghị ọkwa nhazi ọ bụla, yana tiori dị ka dynamical isometry na kernel akwara akwara nke na-ebu amụma ọzụzụ site na mmalite naanị. Mmalite dabere na data, nke na-eme ka akpịrịkpa si n'ihe nlele, bụ ntụzịaka ọzọ na-eto eto.
Mmejuputa n'ezie n'ụwa
A na-ebido CNN nke na-eji ReLU mmegharị ya na mmalite ya nwere nnukwu ụgbọ oloko dị omimi na-enweghị akara efu.
Netwọk nwere arụ ọrụ tanh na-eji mmalite Xavier mee ka ndịiche mgbake kwụsie ike n'ofe ọkwa.
Onye injinia na-ebuli ibu niile na efu na mberede na-ahụ na netwọk anaghị amụta n'ihi na neuron ọ bụla na-adị otu.
Ihe ndabara n'usoro (PyTorch's Kaiming, Keras's Glorot uniform) na-etinye mbido ụkpụrụ na-akpaghị aka mgbe emepụtara oyi akwa.
Ihe ize ndụ & okporo ụzọ nche
Ịkwalite otu akara ngosi nwere ike zoo adịghị ike sistemụ sara mbara.
A na-eledakarị ihe akụrụngwa na ụgwọ ọrụ anya.
Ọdịiche nchekwa na nleba anya nwere ike itolite ka sistemu na-adịwanye mgbagwoju anya.
Map mmejuputa
Kọwaa latency, ịdịmma na ebumnuche ọnụ ahịa tupu mmejuputa ya.
Benchmark n'okpuru ibu dị adị na ọnọdụ data.
Nleba anya akụrụngwa maka mperi, ịkpafu na mmetụta onye ọrụ.
Kwadebe ụzọ nzaghachi azụghachi azụ na ihe omume tupu ịchachaa.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Weight Initialization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Nkezi arọ Stochastic
Ajụjụ a na-ajụkarị
What is Weight Initialization?
Otu esi edobe nha mmalite nke netwọkụ akwara tupu ọzụzụ amalite, nke na-akpụzi ike ma akara na gradients na-adị mma site n'ígwé dị omimi. Mma mmalite dị mma bụ ọdịiche dị n'etiti njikọta ngwa ngwa na ihe nlereanya na-adịghị amụta.
Kedu ihe kpatara ịmalite ibu arọ niile ka ọ bụrụ efu bụ mmejọ na-egbu egbu?
Site na nha nha, neuron ọ bụla na-agbakọ otu mmepụta na gradient, yabụ na ha na-emelite n'otu aka ahụ na oyi akwa enweghị ike ịmụta ihe dị iche iche.
Ọ (Kaiming) mbido ka emebere ya nke ọma maka ọrụ mbido?
Ọ na-ebute ihe dị iche iche site na 2 / fan_in iji kwụọ ụgwọ maka ReLU na-ewepụ ihe dị ka ọkara nke ntinye ya.
Kedu ihe bụ isi ebumnuche nke atụmatụ mbido ibu ibu nke ụkpụrụ?
Atụmatụ dị ka Xavier na Ọ na-ahọrọ ịdị arọ dị iche iche site na nha oyi akwa ka ọ ghara ịla n'iyi ma ọ bụ gbawara ka ha na-agbasa.
Mmalite Xavier (Glorot) dabara nke ọma maka netwọkụ na-eji kedu mmemme?
Xavier na-edozi ọdịiche dị n'ihu na azụ azụ maka symmetric, na-emeju mmemme dị ka tanh na sigmoid.
Kedu ihe kpatara na mmalite ọ na-eji ọdịiche nke 2 / fan_in karịa 1 / fan_in?
ReLU na-agafe naanị ntinye dị mma, na-atụfu ihe dị ka ọkara akara ngosi, ya mere ime mgbanwe abụọ ahụ na-eweghachi mgbanwe mgbanwe a na-atụ anya ya.