Stochastic Gradient Kumanuka hamwe na Momentum
Momentum ni uguhinduranya kumanuka kumanuka ukusanya impuzandengo yikigereranyo cya gradients zashize, ukareka optimizasiyo ikazunguruka vuba binyuze mubibaya no kugabanya ihungabana.
Incamake
It is one of the most widely used training tricks in deep learning.
Kwibira cyane
Ikibaya cya stochastic gradient descente (SGD) ivugurura ibipimo ukandagira mucyerekezo giteganye na mini-batch ya gradient. Ahantu nyaburanga hameze nkimigezi miremire, ifunganye, iyi zig-zag hejuru yinkuta zihanamye mugihe zigenda zinyerera hasi. Momentum, ikunzwe na Polyak nyuma na Rumelhart na bagenzi be, ikosora ibi mukubungabunga umuvuduko wihuta: buri ntambwe ihuza gradient nshya nigice (coefficient deux, akenshi 0.9) yumuvuduko wabanjirije. Icyerekezo gihoraho icyerekezo gishimangira kandi cyihuta, mugihe ibice bihindagurika bihagarika igice. Ikigereranyo cyumubiri ni umupira uremereye umanuka epfo: wubaka umuvuduko muburyo butajegajega kandi ntushobora gutandukanywa n urusaku rwinshi, rutanga guhuza byihuse, byoroshye kuruta vanilla SGD.
Ubushishozi
Ivugurura rituma umuvuduko v uvugururwa nka v = beta * v + gradient, hanyuma ibipimo bigendana no gukuramo igipimo cyo kwiga inshuro v. Hamwe na coefficient beta yumuvuduko, intambwe ifatika mubyerekezo bihamye yongerwaho hafi kubintu 1 / (1 - beta); kuri beta = 0.9 ibyo bikubye inshuro icumi. Iyi ni mibare ifite uburemere buringaniye buringaniye bwa gradients, koroshya urusaku ruto mugihe urinda icyerekezo cyamanuka.
Ingaruka z'Ingamba
Ibyemezo bisobanutse
Iragufasha gutandukanya ibyifuzo bya tekiniki bisobanutse nururimi rwo kwamamaza.
Igiciro na bije
Urashobora kubaza ibibazo byiza byo gushyira mubikorwa mbere yo gukoresha amafaranga cyangwa igihe.
Itsinda hamwe nakazi
Amakipe asangiye ibitekerezo akora ibicuruzwa byiza, politiki, nibyemezo byo kwiga.
Kazoza ka Stochastic Gradient Kumanuka hamwe na Momentum
Momentum ikomeza kuba urufatiro: uburyo bwiza bwo guhuza n'imihindagurikire y'ikirere nka Adam hamwe na variants zayo zashyizemo imbaraga-uburyo bwo kugereranya umwanya-wambere, kandi SGD ifite imbaraga iracyari urufatiro rukomeye rusanzwe rusanzwe rusanzwe rusanzwe kuruta uburyo bwo guhuza n'imihindagurikire y'icyerekezo kinini. Ubushakashatsi burakomeza kuri gahunda yihuta, kugabanuka kubiro, no gukorana kwayo namahugurwa manini cyane. Tegereza imbaraga zo kuguma mubice byingenzi nkuko optimizers ihinduka kuri moderi nini-nini.
Gushyira mu bikorwa Isi
Guhugura imiyoboro yimbitse nka ResNet, aho SGD ifite umuvuduko 0.9 ni resept isanzwe.
Kugereranya urusaku rwinshi rugereranya mugihe ukoresheje mini-mato.
Guhunga ubutumburuke bwibibanza byaho bitwara umuvuduko mukarere.
Gukora nkijambo ryihuta imbere muburyo bwo guhuza n'imikorere nka Adam na RMSprop.
Ingaruka & Kurinda
Amakipe atandukanye arashobora gukoresha ijambo rimwe muburyo butandukanye, sobanura intera hakiri kare.
Ibipimo birashobora kugaragara bikomeye mugihe imikorere-yisi-itaringaniye.
Kwirengagiza ubuziranenge bwamakuru na gahunda yo gusuzuma akenshi bitanga ibisubizo byoroshye.
Igishushanyo mbonera
Tangira nururimi rusobanutse rwibisubizo ukeneye.
Toranya intsinzi imwe hamwe nuburyo bumwe bwo gutsindwa mbere yo kwipimisha.
Koresha umuderevu muto hamwe namakuru ahagarariye, ntabwo ari demo yashizweho.
Inyandiko aho Stochastic Gradient Kumanuka hamwe na Momentum ifasha kandi nuburyo bworoshye bworoshye.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Stochastic Gradient Descent with Momentum quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Kumanuka
Ibibazo bikunze kubazwa
What is Stochastic Gradient Descent with Momentum?
Momentum ni uguhinduranya kumanuka kumanuka ukusanya impuzandengo yikigereranyo cya gradients zashize, ukareka optimizasiyo ikazunguruka vuba binyuze mubibaya no kugabanya ihungabana. Nimwe mumayeri akoreshwa cyane mumahugurwa yimbitse.
Niki ijambo ryihuta ryegeranya mugihe cy'amahugurwa?
Momentum ikomeza umuvuduko wikigereranyo nikigereranyo kiremereye cyimuka cya gradients ziheruka, koroshya icyerekezo cyo kuvugurura.
Mu kibaya kirekire, kigufi, ni ikihe kibazo SGD isanzwe ihura nizo mbaraga zifasha gukemura?
Nta mbaraga, SGD iranyeganyega hejuru yicyerekezo cyikibaya. Momentum ihagarika iyi ihungabana kandi yihuta kuruhande rwikibaya cyiza.
Ni ikihe kigereranyo gifatika gikoreshwa cyane mugusobanura imbaraga?
Momentum igereranwa numupira uremereye wubaka umuvuduko mubyerekezo bihamye kandi ukarwanya gutandukana kuva urusaku.
Nkanswe ni kangahe imbaraga zongerera intambwe ifatika icyerekezo gihamye mugihe beta = 0.9?
Impamvu yo kongera imbaraga ni hafi 1 / (1 - beta), na 1 / (1 - 0.9) = 10, bityo icyerekezo gihoraho cyihuta inshuro icumi.
Ni ubuhe buryo bugezweho bukubiyemo imbaraga-yuburyo bwa mbere?
Adam akomatanya imbaraga zisa-umwanya-wambere (bivuze) igereranya rya gradients hamwe numwanya wa kabiri (variance) igereranya ryo gupima imiterere.