Ubuyobozi bwa tekiniki
Gucisha make
Byoroheje, bikoreshwa cyane kurinda byerekana uburyo bunini bugezweho bushobora kubona mugihe cyamahugurwa.
Umva iki gitaboAI-byavuzwe · ~2 min
Kuriyi page2 min soma
Incamake
It prevents a single huge update from destabilizing or destroying a model, especially in recurrent and language models.
Kwibira cyane
Gukata buhoro buhoro bigabanya ingano ya gradient mbere yuko optimizer ayishyira mu bikorwa. Ifishi isanzwe ni clip-by-norm: ubara L2 igiteranyo cya gradients zose, kandi niba irenze igipimo cyatoranijwe, urapima buri cyiciro munsi kumpamvu imwe kuburyo bisanzwe bingana numubare. Ibi birinda icyerekezo cya update mugihe kigabanya ubunini bwacyo. Impinduka yoroshye, clip-by-agaciro, gusa uhambire buri kintu kigizwe na gradient igizwe murwego rwagenwe nka [-5, 5], ariko irashobora kugoreka icyerekezo cyo kuvugurura. Gukata ni ngombwa muri RNNs na LSTMs, aho ibisasu biturika bikunze kugaragara, kandi ni ikintu cyegereye isi yose muguhugura imiterere yindimi nini, aho rimwe na rimwe ibice bibi cyangwa ibimenyetso bidasanzwe bishobora kubyara igihombo na NaNs.
Gushyira mu bikorwa Isi
Guhugura LSTM kubyara inyandiko, injeniyeri ashyiraho clipnorm = 1.0 kuburyo ibyiciro bidasanzwe biturika ntibibuza kwiga.
Amahugurwa manini yindimi akora hafi ya yose gukuramo clip yisi yose (akenshi kugeza 1.0) kugirango uhoshe igihombo.
DP-SGD ikata urugero rwa buri rugero kurwego rusanzwe mbere yo kongeramo urusaku rwa Gaussiya, rushyira mu bikorwa ingwate itandukanye.
Umwitozo ureba igihombo muri TensorBoard amanura clip clip hanyuma umurongo uhinduka neza kandi uhamye.
Kazoza Kuzuza Gradient
Gukata bikomeza kuba indangagaciro hafi ya buri nini nini yo gutoza imyitozo kuko ihendutse kandi ikomeye. Ubushakashatsi burimo kunonosora hamwe na gahunda yo guhuza n'imihindagurikire y'ikirere ishyiraho urwego rwihuta ruva mu mibare iheruka ya sisitemu aho kuba igiciro gihamye cyagenwe, kandi hamwe na buri cyiciro cyangwa guhuza-gukata neza. Gukata buhoro buhoro kandi bishimangira amahugurwa atandukanye (DP-SGD), aho kurugero rwo gukuramo rugabanya ingaruka za buri cyitegererezo bityo urusaku rwakuweho rushobora kwemeza ubuzima bwite nta nyandiko nimwe yiganjemo icyitegererezo.
Kubishyira mubikorwa
Impamvu ari ngombwa
- Igiciro na bije. Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
- Ibyemezo bisobanutse. Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
- Kugenzura ubuziranenge. Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Witondere
- Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
- Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
- Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Intambwe ikurikira
- Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
- Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
- Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
- Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Ibaruwa y'amakuru y'ubuntu
Ibuka amakuru ya buri munsi ya AI
Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.
Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.
Isuzume
Take the Gradient Clipping quiz
Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.
Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano
Ibibazo bikunze kubazwa
What is Gradient Clipping?
Byoroheje, bikoreshwa cyane kurinda byerekana uburyo bunini bugezweho bushobora kubona mugihe cyamahugurwa. Irinda ivugurura rimwe rinini guhungabanya cyangwa gusenya icyitegererezo, cyane cyane mubyitegererezo byindimi.
Ni ubuhe buryo clip-by-isanzwe igabanya gukuramo cyane cyane kubika mugihe bigabanya ivugurura?
Clip-by-bisanzwe umunzani gradients zose hamwe na scalar imwe, bityo icyerekezo cyo kumanuka kibikwa mugihe uburebure bwintambwe gusa.
Muri clip-by-norm hamwe na bipimo c, bigenda bite mugihe igipimo gisanzwe g_norm kirenze c?
Iyo ibisanzwe ari binini cyane, buri gradient ikurwaho na c / g_norm kuburyo ibisanzwe bivamo bingana numubare c.
Ni ikihe kibazo gukata buhoro buhoro bigenewe kurwanya?
Gukata bigabanya ubunini bwibintu bishya, bikabuza mu buryo butaziguye intambwe nini, zidahindagurika zatewe no guturika gradients.
Nigute clip-by-agaciro itandukaniye na clip-by-bisanzwe?
Clip-by-agaciro ifata buri kintu muburyo bugaragara nka [-5,5]; kuberako ibice byaciwe byigenga, icyerekezo rusange gishobora guhinduka.
A minibatch produces an unusually large global gradient norm. What does clip-by-norm do before the optimizer update?
Norm clipping multiplies all components by a shared factor when the total norm is too large, limiting update magnitude.
Komeza wige
Kuyobora
Abandi bayobozi batoranijwe kuriyi ngingo