Gutangiza ibiro
Nigute washyiraho imiyoboro itangirira uburemere mbere yo gutangira imyitozo, ishushanya cyane niba ibimenyetso na gradients bigumana ubuzima bwiza binyuze mubice byimbitse.
Incamake
Good initialization is the difference between fast convergence and a model that never learns.
Kwibira cyane
Mbere yo kwitoza, uburemere bwose bukenera agaciro ko gutangira. Gushira byose kuri zeru byica: uburemere bumwe butanga gradients imwe, neuron rero ntizigera itandukanya - iki nikibazo cyo guhuza ibice. Gutangiza bisanzwe bisenya guhuza, ariko igipimo gifite akamaro kanini. Nini cyane kandi ibikorwa na gradients biraturika; nto cyane kandi zirashira. Gahunda ngenderwaho ihitamo itandukaniro rishingiye ku bunini bwa layer kugirango igumane ibimenyetso bitandukanye bihoraho murwego rumwe. Xavier (Glorot) itangizwa ryibipimo bitandukanya numubare winjiza wongeyeho ibice bisohoka kandi bikwiranye na tanh na sigmoid. We (Kaiming) umunzani wo gutangiza ukurikije umubare winjiza na konte ya ReLU ikuraho kimwe cya kabiri cyinjiza, bigatuma iba igipimo cyurushundura rwimbitse rushingiye kuri ReLU na CNN. Gutangiza neza bituma imyitozo hakiri kare itajegajega kugeza igihe ibisanzwe hamwe noguhindura imihindagurikire y'ikirere bifata.
Ubushishozi
Intego nugukomeza gutandukanya ibikorwa na gradients bihoraho kuva kumurongo. Xavier ashyiraho uburemere bwibiro kuri 2 / (umufana_in + umufana_out), aringaniza imbere ninyuma yinyuma yibikorwa. Yatangije ikoresha 2 / fan_in kuko ReLU zeru hafi kimwe cya kabiri cyinjiza, bityo gukuba kabiri ibitandukanya byishyura icyo kimenyetso cyatakaye. Kubogama mubisanzwe bitangirwa kuri zeru kuva uburinganire bumaze gucika kuburemere butemewe.
Ingaruka z'Ingamba
Igiciro na bije
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo bisobanutse
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Kugenzura ubuziranenge
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Igihe kizaza cyo gutangiza ibiro
Ibisanzwe bisanzwe hamwe nibisigisigi byasigaye byatumye imyitozo itumva neza itangizwa ryukuri, ariko iracyafite akamaro kumurongo wimbitse cyangwa usanzwe udafite ubusanzwe. Ubushakashatsi bufatika burimo gahunda ijyanye na transformateur no kwitabwaho, uburyo butuma imiyoboro ihugura nta nzego zisanzwe zisanzwe, hamwe na théorie nka isometrie dinamike hamwe na neural tangent kernel iteganya imyitozo kuva yatangira wenyine. Itangiriro-rishingiye ku gutangiza, rihindura umunzani uhereye ku cyitegererezo, ni ikindi cyerekezo gikura.
Gushyira mu bikorwa Isi
CNN ikoresha ibikorwa bya ReLU itangizwa na We yatangije kuburyo bwimbitse bwimyitozo ya gari ya moshi itabuze ibimenyetso.
Urusobe rufite ibikorwa bya tanh rukoresha intangiriro ya Xavier kugirango ibikorwa bitandukana bihamye murwego rumwe.
Injeniyeri utangiza impanuka zose kugeza kuri zeru abona urusobe rwananiwe kwiga kuko buri neuron igumaho.
Imikorere isanzwe (Kaiming ya PyTorch, imyenda ya Glorot ya Keras) ikoresha amahame yo gutangiza mu buryo bwikora mugihe urwego rwaremewe.
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Weight Initialization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Kugereranya Ibiro Bikomeye
Ibibazo bikunze kubazwa
What is Weight Initialization?
Nigute washyiraho imiyoboro itangirira uburemere mbere yo gutangira imyitozo, ishushanya cyane niba ibimenyetso na gradients bigumana ubuzima bwiza binyuze mubice byimbitse. Gutangiza neza ni itandukaniro riri hagati yo guhuza byihuse nicyitegererezo kitigera cyiga.
Kuki gutangiza ibipimo byose kuri zeru ari ikosa ryica?
Hamwe nuburemere bumwe, buri neuron ibara ibisohoka hamwe na gradient imwe, kuburyo bavugurura kimwe kandi urwego ntirushobora kwiga ibintu bitandukanye.
We (Kaiming) itangizwa ryagenewe byumwihariko ibikorwa byo gukora?
Yatangije umunzani utandukanya na 2 / fan_in kugirango yishyure ReLU zeru hafi kimwe cya kabiri cyinjiza.
Niyihe ntego nyamukuru ya gahunda yo gutangiza ibiro?
Imigambi nka Xavier na Yahisemo itandukaniro ryuburemere kuva mubunini kuburyo ibimenyetso bitazimangana cyangwa ngo biturike nkuko bikwirakwiza.
Gutangiza Xavier (Glorot) bikwiranye neza nurusobe ukoresheje ibikorwa?
Xavier iringaniza imbere ninyuma itandukanye kubikorwa, guhuza ibikorwa nka tanh na sigmoid.
Kuki Itangiriro ikoresha itandukaniro rya 2 / umufana_in aho gukoresha 1 / umufana_in?
ReLU inyuramo gusa inyongeramusaruro nziza, ikajugunya hafi kimwe cya kabiri cyibimenyetso, bityo gukuba kabiri ibitandukanya bigarura ibyateganijwe gukora.