Kugereranya Ibiro Bikomeye
Kugereranya Ibiro bya Stochastic (SWA) bifata impuzandengo yoroheje yuburemere bwikitegererezo kuva kumanota menshi yatinze mumyitozo aho gukomeza gufata ifoto yanyuma.
Incamake
This cheap trick often lands the model in a flatter, wider region of the loss landscape, which tends to generalize noticeably better on unseen data.
Kwibira cyane
SWA yatangijwe na Izmailov, Wilson na bagenzi be mu 2018, SWA ikoresha uburyo bwo kubona ko SGD ifite igipimo gihoraho cyangwa cyikurikiranya cyiga idahinduka ku ngingo imwe - irazunguruka ikikije ikibaya kinini, kiringaniye. Aho gutoranya imwe muri izo ngingo zihagarika urusaku, SWA ikoresha igipimo cyo hejuru cyo hejuru (akenshi gihoraho cyangwa cyikurikiranya) cyo kwiga mugihe cyanyuma kandi ugereranije uburemere busura, mubisanzwe buri gihe. Impuzandengo yikigereranyo yicaye hafi yakarere kegeranye. Kuberako ibyiciro-bisanzwe bisanzwe bibarwa kuburemere bwihariye, SWA isaba inzira imwe yinyongera imbere yamakuru kugirango ibaze uburyo BN ikoresha nuburyo butandukanye kuri moderi yagereranijwe. Igiciro ni ubuntu, kandi inyungu zuzuye zirahuza mumashusho yatondekanye ndetse no hanze yacyo.
Ubushishozi
SWA ikomeza ikigereranyo cyo gukora w_SWA = (n · w_SWA + w_i) / (n + 1) yavuguruye buri cyiciro, mugihe moderi ya SGD nzima ikomeza gushakisha hamwe nigipimo kinini cyo kwiga. Ugereranije muburemere bugereranya ensemble mumwanya wimikorere ariko igura icyitegererezo kimwe mubisobanuro, ntabwo ari byinshi. Uburyo bwingenzi ni uko minima iringaniye ifite imbaraga zo guhagarika ibiro, bityo imyitozo / gutakaza ibizamini bigumaho, bikagabanya icyuho rusange.
Ingaruka z'Ingamba
Igiciro na bije
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo bisobanutse
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Kugenzura ubuziranenge
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Igihe kizaza cyo Kuringaniza Ibiro
SWA yabyaye ibintu bitandukanye nka SWA-Gaussian (SWAG) kubihendutse bya Bayesiya bihendutse, kandi igitekerezo cyo kugereranya ubu gishimangira amayeri ya Exponential Moving Average akoreshwa cyane muburyo bwo gukwirakwiza, kwigira wenyine, no kwigana urugero runini. Witege ko impuzandengo y'ibiro igumaho 'ifunguro rya sasita' isanzwe muburyo bwo guhugura, hamwe nubushakashatsi bugera no guhuza imiterere yigenga yigenga (isupu yicyitegererezo) no kunoza kalibrasi hamwe nukuri.
Gushyira mu bikorwa Isi
Kuzamura ibizamini bya ResNet na DenseNet byerekana amashusho kuri CIFAR na ImageNet nta giciro cyinyongera.
SWAG (SWA-Gaussian) itanga igereranya ridashidikanywaho kubigereranyo byerekeranye n'umutekano biturutse kumyitozo imwe.
EMA-yuburemere ituma urusobe rwicyitegererezo mumashanyarazi atanga amashusho nka Diffusion Stable.
Kubaka 'isupu ntangarugero' ugereranije impuzandengo nyinshi zateguwe neza kugirango utezimbere imbaraga utarinze kwitoza.
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Stochastic Weight Averaging quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Gutangiza ibiro
Ibibazo bikunze kubazwa
What is Stochastic Weight Averaging?
Kugereranya Ibiro bya Stochastic (SWA) bifata impuzandengo yoroheje yuburemere bwikitegererezo kuva kumanota menshi yatinze mumyitozo aho gukomeza gufata ifoto yanyuma. Aya mayeri ahendutse akenshi yerekana icyitegererezo muburyo bushimishije, mugari mugace k'igihombo, gikunda kuba rusange muburyo bugaragara kumibare itagaragara.
Kugereranya Uburemere bwa Stochastique ni ikihe?
SWA igereranya ibipimo by'icyitegererezo (uburemere) byakusanyirijwe kuri bariyeri nyinshi mugice cyanyuma cyamahugurwa, ntabwo ari ibyahanuwe cyangwa gradients.
Kuki SWA ikunda kunoza rusange?
Impuzandengo yimura igisubizo cyerekeza hagati yakarere kegeranye k’ubutaka bwatakaye, kandi minima iringaniye muri rusange kuko gari ya moshi nigihombo cyibizamini bikomeza guhuza.
Ni izihe ntambwe yinyongera SWA isaba imiyoboro ikoresha ibice bisanzwe?
Kuberako imibare ya BN iterwa nuburemere bwihariye, impuzandengo yikigereranyo ikenera pass imwe yinyongera hejuru yamakuru kugirango ibaze uburyo bwo gukora nuburyo butandukanye.
Ni iyihe myitwarire yo kwiga-isanzwe ikoreshwa mugihe cyo kugereranya SWA?
SWA ikomeza igipimo giciriritse gihoraho cyangwa cyikigero cyo kwiga kuburyo SGD ikomeje gushakisha akarere kagari kangana amanota yabo agereranijwe.
Nigute ibiciro bya SWA byagereranijwe ugereranije na gakondo ya moderi ya N?
SWA isenya ibirindiro byinshi murwego rumwe rwagereranijwe, bityo rero umwanzuro ugura kimwe na moderi imwe aho kuba N.