Gucisha make
Byoroheje, bikoreshwa cyane kurinda byerekana uburyo bunini bugezweho bushobora kubona mugihe cyamahugurwa.
Incamake
It prevents a single huge update from destabilizing or destroying a model, especially in recurrent and language models.
Kwibira cyane
Gukata buhoro buhoro bigabanya ingano ya gradient mbere yuko optimizer ayishyira mu bikorwa. Ifishi isanzwe ni clip-by-norm: ubara L2 igiteranyo cya gradients zose, kandi niba irenze igipimo cyatoranijwe, urapima buri cyiciro munsi kumpamvu imwe kuburyo bisanzwe bingana numubare. Ibi birinda icyerekezo cya update mugihe kigabanya ubunini bwacyo. Impinduka yoroshye, clip-by-agaciro, gusa uhambire buri kintu kigizwe na gradient igizwe murwego rwagenwe nka [-5, 5], ariko irashobora kugoreka icyerekezo cyo kuvugurura. Gukata ni ngombwa muri RNNs na LSTMs, aho ibisasu biturika bikunze kugaragara, kandi ni ikintu cyegereye isi yose muguhugura imiterere yindimi nini, aho rimwe na rimwe ibice bibi cyangwa ibimenyetso bidasanzwe bishobora kubyara igihombo na NaNs.
Ubushishozi
Muri clip-by-norm, ubara g_norm, L2 igipimo cya gradient ya vector. Niba g_norm irenze igipimo c, ugwiza buri cyiciro na c / g_norm; bitabaye ibyo ukabasiga bidahindutse. Kuberako upima ibice byose ukoresheje scalar imwe, icyerekezo cyo kumanuka kirabitswe kandi uburebure bwintambwe gusa. Clip-by-agaciro ifata buri kintu cyigenga, gishobora guhindura icyerekezo ariko kigahuzwa neza buri kintu cyose.
Ingaruka z'Ingamba
Igiciro na bije
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo bisobanutse
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Kugenzura ubuziranenge
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Kazoza Kuzuza Gradient
Gukata bikomeza kuba indangagaciro hafi ya buri nini nini yo gutoza imyitozo kuko ihendutse kandi ikomeye. Ubushakashatsi burimo kunonosora hamwe na gahunda yo guhuza n'imihindagurikire y'ikirere ishyiraho urwego rwihuta ruva mu mibare iheruka ya sisitemu aho kuba igiciro gihamye cyagenwe, kandi hamwe na buri cyiciro cyangwa guhuza-gukata neza. Gukata buhoro buhoro kandi bishimangira amahugurwa atandukanye (DP-SGD), aho kurugero rwo gukuramo rugabanya ingaruka za buri cyitegererezo bityo urusaku rwakuweho rushobora kwemeza ubuzima bwite nta nyandiko nimwe yiganjemo icyitegererezo.
Gushyira mu bikorwa Isi
Guhugura LSTM kubyara inyandiko, injeniyeri ashyiraho clipnorm = 1.0 kuburyo ibyiciro bidasanzwe biturika ntibibuza kwiga.
Amahugurwa manini yindimi akora hafi ya yose gukuramo clip yisi yose (akenshi kugeza 1.0) kugirango uhoshe igihombo.
DP-SGD ikata urugero rwa buri rugero kurwego rusanzwe mbere yo kongeramo urusaku rwa Gaussiya, rushyira mu bikorwa ingwate itandukanye.
Umwitozo ureba igihombo muri TensorBoard amanura clip clip hanyuma umurongo uhinduka neza kandi uhamye.
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Gradient Clipping quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Kugenzura Gradient
Ibibazo bikunze kubazwa
What is Gradient Clipping?
Byoroheje, bikoreshwa cyane kurinda byerekana uburyo bunini bugezweho bushobora kubona mugihe cyamahugurwa. Irinda ivugurura rimwe rinini guhungabanya cyangwa gusenya icyitegererezo, cyane cyane mubyitegererezo byindimi.
Ni ubuhe buryo clip-by-isanzwe igabanya gukuramo cyane cyane kubika mugihe bigabanya ivugurura?
Clip-by-bisanzwe umunzani gradients zose hamwe na scalar imwe, bityo icyerekezo cyo kumanuka kibikwa mugihe uburebure bwintambwe gusa.
Muri clip-by-norm hamwe na bipimo c, bigenda bite mugihe igipimo gisanzwe g_norm kirenze c?
Iyo ibisanzwe ari binini cyane, buri gradient ikurwaho na c / g_norm kuburyo ibisanzwe bivamo bingana numubare c.
Ni ikihe kibazo gukata buhoro buhoro bigenewe kurwanya?
Gukata bigabanya ubunini bwibintu bishya, bikabuza mu buryo butaziguye intambwe nini, zidahindagurika zatewe no guturika gradients.
Nigute clip-by-agaciro itandukaniye na clip-by-bisanzwe?
Clip-by-agaciro ifata buri kintu muburyo bugaragara nka [-5,5]; kuberako ibice byaciwe byigenga, icyerekezo rusange gishobora guhinduka.
Ni ukubera iki gukata buhoro buhoro kwisi yose mumahugurwa manini yindimi?
Ku bunini bunini, inyongeramusaruro zidasanzwe zishobora gutanga gradients nini; gukuramo amahame yisi yose bituma iyi mitwe idahungabanya kwiruka.