AMABWIRIZA Yibanze
Stochastic Gradient Kumanuka hamwe na Momentum
Momentum ni uguhinduranya kumanuka kumanuka ukusanya impuzandengo yikigereranyo cya gradients zashize, ukareka optimizasiyo ikazunguruka vuba binyuze mubibaya no kugabanya ihungabana.
Umva iki gitaboAI-byavuzwe · ~2 min
Kuriyi page2 min soma
Incamake
Nimwe mumayeri akoreshwa cyane mumahugurwa yimbitse.
Kwibira cyane
Ikibaya cya stochastic gradient descente (SGD) ivugurura ibipimo ukandagira mucyerekezo giteganye na mini-batch ya gradient. Ahantu nyaburanga hameze nkimigezi miremire, ifunganye, iyi zig-zag hejuru yinkuta zihanamye mugihe zigenda zinyerera hasi. Momentum, ikunzwe na Polyak nyuma na Rumelhart na bagenzi be, ikosora ibi mukubungabunga umuvuduko wihuta: buri ntambwe ihuza gradient nshya nigice (coefficient deux, akenshi 0.9) yumuvuduko wabanjirije. Icyerekezo gihoraho icyerekezo gishimangira kandi cyihuta, mugihe ibice bihindagurika bihagarika igice. Ikigereranyo cyumubiri ni umupira uremereye umanuka epfo: wubaka umuvuduko muburyo butajegajega kandi ntushobora gutandukanywa n urusaku rwinshi, rutanga guhuza byihuse, byoroshye kuruta vanilla SGD.
Gushyira mu bikorwa Isi
Guhugura imiyoboro yimbitse nka ResNet, aho SGD ifite umuvuduko 0.9 ni resept isanzwe.
Kugereranya urusaku rwinshi rugereranya mugihe ukoresheje mini-mato.
Guhunga ubutumburuke bwibibanza byaho bitwara umuvuduko mukarere.
Gukora nkijambo ryihuta imbere muburyo bwo guhuza n'imikorere nka Adam na RMSprop.
Kazoza ka Stochastic Gradient Kumanuka hamwe na Momentum
Momentum ikomeza kuba urufatiro: uburyo bwiza bwo guhuza n'imihindagurikire y'ikirere nka Adam hamwe na variants zayo zashyizemo imbaraga-uburyo bwo kugereranya umwanya-wambere, kandi SGD ifite imbaraga iracyari urufatiro rukomeye rusanzwe rusanzwe rusanzwe rusanzwe kuruta uburyo bwo guhuza n'imihindagurikire y'icyerekezo kinini. Ubushakashatsi burakomeza kuri gahunda yihuta, kugabanuka kubiro, no gukorana kwayo namahugurwa manini cyane. Tegereza imbaraga zo kuguma mubice byingenzi nkuko optimizers ihinduka kuri moderi nini-nini.
Kubishyira mubikorwa
Impamvu ari ngombwa
- Ibyemezo bisobanutse. Iragufasha gutandukanya ibyifuzo bya tekiniki bisobanutse nururimi rwo kwamamaza.
- Igiciro na bije. Urashobora kubaza ibibazo byiza byo gushyira mubikorwa mbere yo gukoresha amafaranga cyangwa igihe.
- Itsinda hamwe nakazi. Amakipe asangiye ibitekerezo akora ibicuruzwa byiza, politiki, nibyemezo byo kwiga.
Witondere
- Amakipe atandukanye arashobora gukoresha ijambo rimwe muburyo butandukanye, sobanura intera hakiri kare.
- Ibipimo birashobora kugaragara bikomeye mugihe imikorere-yisi-itaringaniye.
- Kwirengagiza ubuziranenge bwamakuru na gahunda yo gusuzuma akenshi bitanga ibisubizo byoroshye.
Intambwe ikurikira
- Tangira nururimi rusobanutse rwibisubizo ukeneye.
- Toranya intsinzi imwe hamwe nuburyo bumwe bwo gutsindwa mbere yo kwipimisha.
- Koresha umuderevu muto hamwe namakuru ahagarariye, ntabwo ari demo yashizweho.
- Inyandiko aho Stochastic Gradient Kumanuka hamwe na Momentum ifasha kandi nuburyo bworoshye bworoshye.
Komeza Ubushakashatsi
Ibaruwa y'amakuru y'ubuntu
Ibuka amakuru ya buri munsi ya AI
Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.
Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.
Isuzume
Take the Stochastic Gradient Descent with Momentum quiz
Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.
Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano
Ibibazo bikunze kubazwa
Niki Kumanuka kwa Stochastic Gradient hamwe na Momentum?
Momentum ni uguhinduranya kumanuka kumanuka ukusanya impuzandengo yikigereranyo cya gradients zashize, ukareka optimizasiyo ikazunguruka vuba binyuze mubibaya no kugabanya ihungabana. Nimwe mumayeri akoreshwa cyane mumahugurwa yimbitse.
Niki ijambo ryihuta ryegeranya mugihe cy'amahugurwa?
Momentum ikomeza umuvuduko wikigereranyo nikigereranyo kiremereye cyimuka cya gradients ziheruka, koroshya icyerekezo cyo kuvugurura.
Mu kibaya kirekire, kigufi, ni ikihe kibazo SGD isanzwe ihura nizo mbaraga zifasha gukemura?
Nta mbaraga, SGD iranyeganyega hejuru yicyerekezo cyikibaya. Momentum ihagarika iyi ihungabana kandi yihuta kuruhande rwikibaya cyiza.
Ni ikihe kigereranyo gifatika gikoreshwa cyane mugusobanura imbaraga?
Momentum igereranwa numupira uremereye wubaka umuvuduko mubyerekezo bihamye kandi ukarwanya gutandukana kuva urusaku.
Nkanswe ni kangahe imbaraga zongerera intambwe ifatika icyerekezo gihamye mugihe beta = 0.9?
Impamvu yo kongera imbaraga ni hafi 1 / (1 - beta), na 1 / (1 - 0.9) = 10, bityo icyerekezo gihoraho cyihuta inshuro icumi.
Ni ubuhe buryo bugezweho bukubiyemo imbaraga-yuburyo bwa mbere?
Adam akomatanya imbaraga zisa-umwanya-wambere (bivuze) igereranya rya gradients hamwe numwanya wa kabiri (variance) igereranya ryo gupima imiterere.
Komeza wige
Kuyobora
Abandi bayobozi batoranijwe kuriyi ngingo