Bacewa da Fashe Gradients
Lokacin horar da cibiyoyin sadarwa masu zurfi, siginonin kuskure suna raguwa zuwa sifili ko busawa zuwa ga iyaka yayin da suke tafiya baya ta yadudduka da yawa.
Dubawa
This makes deep and recurrent models painfully slow or impossible to train without specific fixes.
Zurfafa nutsewa
Cibiyoyin jijiyoyi suna koyo ta hanyar yaɗa baya, wanda ke ninka Layer Layer ta Layer ta amfani da tsarin sarkar. Lokacin da kuka tara yadudduka da yawa, waɗannan abubuwan na kowane Layer suna ninka tare. Idan kowane ma'auni ya kasance ƙasa da 1 akai-akai, samfurin yana raguwa sosai kuma farkon yadudduka ba sa sabuntawa - matsalar gradient mai ɓacewa. Idan kowane ma'auni ya fi 1 girma, samfurin ya fashe, yana haifar da ɗaukakawa mara ƙarfi ko ƙimar NaN. Ayyuka masu gamsarwa kamar sigmoid da tanh, waɗanda abubuwan da suka samo asali sun fi girma a 0.25 da 1, manyan masu laifi ne. Batun ya fi tsanani a cikin hanyoyin sadarwa mai zurfi (RNNs) masu sarrafa dogayen jeri, inda ake sake maimaita matrix ɗin nauyi iri ɗaya a kowane lokaci, yana haɓaka tasirin sosai.
Fahimtar Fasaha
A cikin yaɗuwar gradient a farkon Layer samfuri ne na yawancin sharuɗɗan Yakubu da nauyi. Kusan, siginar tana yin ma'auni kamar ma'aunin kowane Layer wanda aka ɗaga zuwa zurfin. Ƙimar da ke ƙarƙashin lalacewa 1 zuwa sifili; dabi'u sama da 1 suna girma ba tare da daure ba. Ga RNN da ba a binne sama da matakan T ba, mafi girman lokacin yana yin kama da ma'aunin nauyi na yau da kullun zuwa ikon T, don haka ko da ƙananan karkata daga 1 suna ɓacewa ko fashe cikin dogon jerin.
Dabarun Tasiri
Kudin da kasafin kuɗi
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Shawarwari masu haske
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Kula da inganci
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Makomar Bacewa da Fashe Gradients
Babban ragi - haɗin kai na saura (tsalle), daidaitawa, gating, da farawa a hankali - yanzu daidaitattun su ne, don haka ɓatawar gradients da wuya su hana horar da gine-ginen zamani. Masu canji suna karkatar da haɗaɗɗun maimaitawa gaba ɗaya ta hanyar yin amfani da hankali akan jeri maimakon maimaita maimaitawa na matrix ɗaya. Ana ci gaba da bincike akan hanyoyin sadarwar horarwa dubban yadudduka masu zurfi, akan samfuran yanayi masu tsayin gaske, da kuma kan kayan aikin ka'idoji kamar kernel na jijiyoyi waɗanda ke hasashen yaduwar sigina kafin matakin horo ɗaya ya gudana.
Aiwatar da Gaskiyar Duniya
Samfuran yaren RNN na farko sun yi gwagwarmayar haɗa kalmomi a cikin dogon jimla saboda gradients sun ɓace a lokuta da yawa, suna ƙarfafa LSTMs da GRUs.
ResNet ya ba da damar horar da rarrabuwa na hoto Layer 100+ ta ƙara haɗin tsallake-tsallake waɗanda ke ba gradients hanya kai tsaye, mara diluted baya.
Mai haɓakawa yana ganin asarar horo ba zato ba tsammani ya zama NaN - alama ce ta fashe gradients - kuma yana ƙara yankan gradient don daidaita shi.
Kayan aikin sa ido a cikin PyTorch ko TensorFlow mãkirci kowane-Layer gradient ka'idoji don injiniyoyi su iya hango wani Layer wanda gradients ɗinsa ya rushe zuwa kusa da sifili.
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Vanishing and Exploding Gradients quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Duban hankali na Gradient
Tambayoyin da ake yawan yi
What is Vanishing and Exploding Gradients?
Lokacin horar da cibiyoyin sadarwa masu zurfi, siginonin kuskure suna raguwa zuwa sifili ko busawa zuwa ga iyaka yayin da suke tafiya baya ta yadudduka da yawa. Wannan yana sa ƙirar ƙira mai zurfi da maimaituwa jinkirin jinkiri ko rashin yuwuwar horarwa ba tare da takamaiman gyara ba.
Wane aiki na lissafin lissafi a baya shine tushen ɓata da fashewar gradients?
Bayar da baya yana amfani da tsarin sarkar, yana ninka abubuwa da yawa na kowane Layer tare; samfuran ƙima a ƙarƙashin 1 sun ɓace kuma samfuran sama da 1 sun fashe.
Me yasa sigmoid da tanh kunnawa ke da wuya musamman ga bacewar gradients?
Sigmoid ta kololuwa a 0.25 da tanh ta 1; A cikin madaidaitan yankuna duka biyun suna kusantar sifili, don haka tara su yana fitar da gradients zuwa sifili.
Wanne gine-gine ne ya fi shafan matsalolin gradient a kan dogon jerin abubuwa?
RNN yana sake amfani da matrix mai maimaita nauyi iri ɗaya a kowane lokaci, don haka a cikin dogon jerin abubuwan tasirin tasirin kamar wannan ƙimar matrix ɗin da aka ɗaga zuwa tsayin jeri.
Ganin asarar horo ba zato ba tsammani ya juya zuwa NaN mai yiwuwa ya nuna wace matsala?
Fashe gradients suna samar da babban sabuntawa wanda ke zubewa zuwa iyaka ko NaN; bacewar gradients maimakon haka yana haifar da asara ta tsaya cak.
Ta yaya hanyoyin haɗin gwiwa (tsalle) ke taimakawa tare da bacewar gradients?
Tsallake haɗin kai yana ƙara hanyar ainihi don haka gradients za su iya komawa baya ba tare da an rage su akai-akai ta hanyar tsaka-tsaki ba.