Saukowar Gradient Stochastic tare da Momentum
Momentum wani tweak ne zuwa zuriyar gradient wanda ke tara matsakaita masu gudana na gradients da suka gabata, yana barin haɓakawa da sauri ta birgima cikin kwaruruka da daskarewa juzu'i.
Dubawa
It is one of the most widely used training tricks in deep learning.
Zurfafa nutsewa
Zauren zuriyar stochastic gradient (SGD) yana sabunta sigogi ta hanyar tafiya a gaba da ƙaramin gradient na yanzu. A cikin shimfidar wurare masu siffa kamar dogayen raƙuman kunkuntar, wannan zigzags a kan bangon tudu yayin da yake rarrafe tare da ƙasa mai laushi. Momentum, wanda Polyak ya shahara kuma daga baya ta Rumelhart da abokan aiki, yana gyara wannan ta hanyar kiyaye saurin motsi: kowane mataki yana haɗa sabon gradient tare da ɗan juzu'i (madaidaicin lokaci, sau da yawa 0.9) na saurin baya. Madaidaicin kwatancen gradient yana ƙarfafawa da haɓakawa, yayin da abubuwan da ke jujjuyawa sun soke wani ɓangare. Kwatankwacin jiki wani nauyi ne mai nauyi wanda ke jujjuyawa ƙasa: yana haɓaka sauri a madaidaiciyar kwatance kuma ba a iya jujjuya shi ta hanyar ƙarar hayaniya, yana ba da sauri, daidaitawa mai santsi fiye da vanilla SGD.
Fahimtar Fasaha
Sabuntawa yana kiyaye saurin v wanda aka sabunta azaman v = beta * v + gradient, sannan sigogi suna motsawa ta rage lokutan koyo v. Tare da ingantaccen beta na lokaci-lokaci, ingantaccen mataki a cikin madaidaiciyar shugabanci yana ƙaruwa da ƙima ta hanyar 1/(1 - beta); a beta = 0.9 wato kusan sau goma. Wannan a ilimin lissafi matsakaicin matsakaicin nauyi mai nauyi ne na gradients, yana daidaita ƙaramar ƙaramar ƙara yayin da yake kiyaye babbar hanyar gangara.
Dabarun Tasiri
Shawarwari masu haske
Yana taimaka muku keɓance bayyanannen da'awar fasaha daga harshen talla.
Kudin da kasafin kuɗi
Kuna iya yin mafi kyawun tambayoyin aiwatarwa kafin kashe kuɗi ko lokaci.
Ƙungiya da aikin aiki
Ƙungiyoyin da ke da fahimtar juna suna yin mafi kyawun samfura, manufofi, da yanke shawara na koyo.
Makomar Stochastic Gradient Deescent tare da Momentum
Ƙaddamarwa ya kasance mai tushe: masu haɓaka masu daidaitawa kamar Adam da bambance-bambancen sa sun haɗa ƙima mai ƙima na lokacin farko, kuma SGD tare da ƙwaƙƙwaran har yanzu ƙaƙƙarfan tushen tushe ne wanda galibi ke haɓaka mafi kyawun hanyoyin daidaitawa akan manyan samfuran hangen nesa. Ana ci gaba da bincike kan jadawalin lokaci mai ƙarfi, ruɓar nauyin nauyi, da hulɗar sa tare da manyan horarwa. Yi tsammanin zazzaɓi zai kasance babban sashi yayin da masu haɓakawa ke tasowa don ƙila mafi girma.
Aiwatar da Gaskiyar Duniya
Horar da zurfafa hanyoyin sadarwa kamar ResNet, inda SGD tare da lokacin 0.9 shine madaidaicin girke-girke.
Ƙididdiga masu laushi masu laushi lokacin amfani da ƙananan ƙananan batches.
Gudun tudun mun tsira ta hanyar ɗaukar sauri ta yankuna masu faɗi.
Yin hidima azaman lokacin ɗorewa a cikin abubuwan ingantawa kamar Adam da RMSprop bambance-bambancen.
Hatsari & Tsare-tsare
Ƙungiyoyi daban-daban na iya amfani da kalmar iri ɗaya daban, don haka ayyana iyaka da wuri.
Alamomi na iya yin kama da ƙarfi yayin da aikin zahirin duniya bai yi daidai ba.
Yin watsi da ingancin bayanai da tsare-tsaren kimantawa galibi yana haifar da sakamako mara ƙarfi.
Taswirar Hanya
Fara da ma'anar harshe a sarari na sakamakon da kuke buƙata.
Zaɓi ma'aunin nasara ɗaya da yanayin gazawa ɗaya kafin gwaji.
Gudun ƙaramin matukin jirgi tare da bayanan wakilci, ba saitin demo da aka goge ba.
Daftarin aiki inda Stochastic Gradient Deescent tare da Momentum yana taimakawa kuma inda mafi sauƙi hanyoyin suka fi kyau.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Stochastic Gradient Descent with Momentum quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Saukowar Gradient
Tambayoyin da ake yawan yi
What is Stochastic Gradient Descent with Momentum?
Momentum wani tweak ne zuwa zuriyar gradient wanda ke tara matsakaita masu gudana na gradients da suka gabata, yana barin haɓakawa da sauri ta birgima cikin kwaruruka da daskarewa juzu'i. Yana daya daga cikin dabarun horarwa da aka fi amfani da shi wajen zurfafa ilmantarwa.
Menene lokacin ƙarfin aiki ke tarawa yayin horo?
Momentum yana kula da motsi mai saurin gudu wanda shine matsakaicin matsakaicin nauyi mai nauyi na kwanan nan, yana daidaita alkiblar sabuntawa.
A cikin dogo, kunkuntar kwazazzabo, wace matsala ce a sarari SGD ke fama da wannan motsi yana taimakawa gyara?
Ba tare da ɓata lokaci ba, SGD yana jujjuyawa ƙetare manyan kwatance na kwazazzabo. Momentum yana soke waɗannan jujjuyawar kuma yana haɓaka tare da benen kwari mai laushi.
Wane kwatankwacin jiki ne aka fi amfani da shi don kwatanta ƙarfin hali?
Ana kamanta lokacin da ƙwallon ƙafa mai nauyi wanda ke haɓaka gudu a daidaitattun kwatance kuma yana ƙin jujjuyawa daga karan hayaniya.
Kusan nawa ne kuzari ke haɓaka ingantaccen mataki a madaidaiciyar hanya lokacin beta = 0.9?
Matsakaicin haɓakawa shine kusan 1/(1 - beta), da 1/(1 - 0.9) = 10, don haka daidaitattun kwatance ana haɓaka kusan ninki goma.
Wanne ingantawa na zamani ya haɗa da ƙiyasin salon-ƙara?
Adamu ya haɗu da ƙima-kamar farkon-lokacin (ma'ana) ƙiyasin gradients tare da ƙiyasin lokaci na biyu (bambance-bambancen) don daidaita sikeli.