Gradients na-apụ n'anya ma na-agbawa
Mgbe ị na-azụ netwọk dị omimi, akara njehie na-adaba na efu ma ọ bụ na-efe efe ruo enweghi ngwụcha ka ha na-aga azụ azụ n'ọtụtụ ọkwa.
Nchịkọta
This makes deep and recurrent models painfully slow or impossible to train without specific fixes.
Ime miri emi
Netwọk akwara na-amụta site n'ịgbasa azụ, nke na-amụba oyi akwa gradients site na iji usoro agbụ. Mgbe ị na-achịkọta ọtụtụ n'ígwé, ihe ndị ahụ n'otu-layer na-amụba ọnụ. Ọ bụrụ na ihe nke ọ bụla na-erughị 1, ngwaahịa ahụ na-adalata nke ukwuu na ọkwa mbụ anaghị emelite - nsogbu gradient na-apụ n'anya. Ọ bụrụ na ihe nke ọ bụla karịrị 1, ngwaahịa a na-agbawa, na-emepụta nnukwu mmelite na-akwụghị ụgwọ ma ọ bụ ụkpụrụ NaN. Nrụ ọrụ na-eju afọ dị ka sigmoid na tanh, ndị ihe nrụpụta ha kacha na 0.25 na 1, bụ ndị omekome mara mma. Okwu a kacha njọ na ụgbụ ntanye nke miri emi yana na netwọkụ na-emegharị ugboro ugboro (RNNs) na-ahazi usoro ogologo oge, ebe a na-emegharị otu matrix dị arọ ahụ n'oge ọ bụla, na-eme ka mmetụta ahụ dịkwuo elu.
Nghọta nka nka
N'ịgbasa gradient na mbụ oyi akwa bụ ngwaahịa nke ọtụtụ okwu Jacobian na arọ. N'ihe dị ka ọ dị, mgbama ahụ na-atụ ihe dị ka ihe na-akpata kwa oyi akwa ewelitere ruo omimi. Uru n'okpuru 1 ire ere gaa na efu; ụkpụrụ karịrị 1 na-eto na-enweghị oke. Maka RNN ewepụghị n'elu usoro T, okwu a na-achị na-eme dị ka uru nke ịdị arọ na-emegharị ugboro ugboro ruo na ike T, ya mere, ọbụna obere ngbanwe site na 1 na-apụ n'anya ma ọ bụ gbawara ogologo usoro.
Mmetụta atụmatụ
Ọnụ ego na mmefu ego
Mkpebi ihe owuwu ụlọ na-akwalite arụmọrụ yana ọnụ ahịa ọrụ ruo ọtụtụ afọ.
Mkpebi doro anya
Nkà mmụta nka na-enyere ndị otu egwuregwu aka ịhọrọ nchịkọta ziri ezi, ọ bụghị naanị nke kachasị ọhụrụ.
Quality akara
Nhọrọ injinia ka mma na-ebelata ihe omume ntụkwasị obi na mmepụta.
Ọdịnihu nke ịla n'iyi na mgbawa gradients
Mbelata ndị bụ isi - njikọ ndị fọdụrụ (ịwụpụ), nhazigharị, ịgba egwu, na ịkpachapụ anya mmalite - bụzi ọkọlọtọ, yabụ gradients na-apụ n'anya anaghị egbochi ọzụzụ nke ụlọ ọgbara ọhụrụ. Ndị na-agbanwe agbanwe na-ewepụ ihe mgbagwoju anya na-emegharị ugboro ugboro kpamkpam site n'iji nlebara anya n'usoro n'usoro kama ịmegharịgharị otu matriks ugboro ugboro. Nnyocha na-aga n'ihu na netwọk ọzụzụ ọtụtụ puku ọkwa miri emi, na ụdị ọnọdụ dị ogologo ogologo oge, yana na ngwaọrụ usoro dị ka kernel akwara akwara nke na-ebu amụma mgbasa mgbaàmà tupu otu usoro ọzụzụ ga-aga.
Mmejuputa n'ezie n'ụwa
Ụdị asụsụ RNN mbụ gbalịsiri ike ijikọ okwu n'ofe ahịrịokwu dị ogologo n'ihi na gradients na-apụ n'anya n'ọtụtụ oge, na-akpali LSTM na GRU.
ResNet nyere ọzụzụ nke nhazi ihe onyonyo oyi akwa 100+ site na ịgbakwunye njikọ mwụda na-enye gradients ụzọ kwụ ọtọ, enweghị mgbagha azụ.
Onye nrụpụta na-ahụ mfu ọzụzụ na mberede na-aghọ NaN - akara ngosi nke gradients na-agbawa - ma na-agbakwunye gradient clipping iji mee ka ọ kwụsie ike.
Ngwaọrụ nlekota na PyTorch ma ọ bụ TensorFlow plot per-layer gradient norms ka ndị injinia nwee ike ịhụ oyi akwa nke gradients ya daa n'akụkụ efu.
Ihe ize ndụ & okporo ụzọ nche
Ịkwalite otu akara ngosi nwere ike zoo adịghị ike sistemụ sara mbara.
A na-eledakarị ihe akụrụngwa na ụgwọ ọrụ anya.
Ọdịiche nchekwa na nleba anya nwere ike itolite ka sistemu na-adịwanye mgbagwoju anya.
Map mmejuputa
Kọwaa latency, ịdịmma na ebumnuche ọnụ ahịa tupu mmejuputa ya.
Benchmark n'okpuru ibu dị adị na ọnọdụ data.
Nleba anya akụrụngwa maka mperi, ịkpafu na mmetụta onye ọrụ.
Kwadebe ụzọ nzaghachi azụghachi azụ na ihe omume tupu ịchachaa.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Vanishing and Exploding Gradients quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Ntụlegharị gradient
Ajụjụ a na-ajụkarị
What is Vanishing and Exploding Gradients?
Mgbe ị na-azụ netwọk dị omimi, akara njehie na-adaba na efu ma ọ bụ na-efe efe ruo enweghi ngwụcha ka ha na-aga azụ azụ n'ọtụtụ ọkwa. Nke a na-eme ka ụdị dị omimi na nke na-eme ugboro ugboro na-egbu mgbu ma ọ bụ na-agaghị ekwe omume ịzụ ya na-enweghị nhazi ụfọdụ.
Kedu ọrụ mgbakọ na mwepụ na mgbasa ozi na-akpata bụ isi ihe na-apụ n'anya na mgbawa gradients?
Backpropagation na-emetụta usoro agbụ, na-amụba ọtụtụ ihe kwa-layer ọnụ; ngwaahịa nke ụkpụrụ n'okpuru 1 na-apụ n'anya na ngwaahịa karịrị 1 na-agbawa.
Kedu ihe kpatara sigmoid na tanh ji adịkarị mfe na gradients na-apụ n'anya?
Ọnụ ọgụgụ kasị elu nke Sigmoid na 0.25 na tanh na 1; N'ime mpaghara juru eju, ha abụọ na-abịaru nso efu, yabụ ịdebe ha na-ebuga gradients gaa na efu.
Kedu ihe owuwu ụlọ ka nsogbu gradient kacha emetụta n'ime ogologo usoro?
Otu RNN na-etinyekwa otu matriks ịdị arọ na-emegharị ugboro ugboro n'oge ọ bụla, yabụ n'ime ogologo usoro, mmetụta ogige dị ka eigenvalue matriks ahụ welitere n'ogologo usoro.
Ịhụ nkwụsị ọzụzụ na-atụgharị na mberede na NaN nwere ike igosi nsogbu dị?
gradients na-agbawa agbawa na-emepụta mmelite buru ibu na-ejupụta na enweghi ngwụcha ma ọ bụ NaN; gradients na-apụ n'anya kama na-eme ka mfu kwụsị.
Kedu ka njikọ fọdụrụ (ịwụpụ) si enyere aka na gradients na-apụ n'anya?
Njikọ mwụpụ na-agbakwụnye ụzọ njirimara ka gradients wee nwee ike ịla azụ na-enweghị nleda anya ugboro ugboro site na ọkwa etiti.