Ubuyobozi bwa tekiniki

Kugereranya Ibiro Bikomeye

Kugereranya Ibiro bya Stochastic (SWA) bifata impuzandengo yoroheje yuburemere bwikitegererezo kuva kumanota menshi yatinze mumyitozo aho gukomeza gufata ifoto yanyuma.

Umva iki gitaboAI-byavuzwe · ~2 min

Kuriyi page2 min soma
  1. Incamake
  2. Kwibira cyane
  3. Gushyira mu bikorwa Isi
  4. Igihe kizaza cyo Kuringaniza Ibiro
  5. Kubishyira mubikorwa
  6. Komeza Ubushakashatsi
  7. Ibibazo bikunze kubazwa

Incamake

This cheap trick often lands the model in a flatter, wider region of the loss landscape, which tends to generalize noticeably better on unseen data.

Kwibira cyane

SWA yatangijwe na Izmailov, Wilson na bagenzi be mu 2018, SWA ikoresha uburyo bwo kubona ko SGD ifite igipimo gihoraho cyangwa cyikurikiranya cyiga idahinduka ku ngingo imwe - irazunguruka ikikije ikibaya kinini, kiringaniye. Aho gutoranya imwe muri izo ngingo zihagarika urusaku, SWA ikoresha igipimo cyo hejuru cyo hejuru (akenshi gihoraho cyangwa cyikurikiranya) cyo kwiga mugihe cyanyuma kandi ugereranije uburemere busura, mubisanzwe buri gihe. Impuzandengo yikigereranyo yicaye hafi yakarere kegeranye. Kuberako ibyiciro-bisanzwe bisanzwe bibarwa kuburemere bwihariye, SWA isaba inzira imwe yinyongera imbere yamakuru kugirango ibaze uburyo BN ikoresha nuburyo butandukanye kuri moderi yagereranijwe. Igiciro ni ubuntu, kandi inyungu zuzuye zirahuza mumashusho yatondekanye ndetse no hanze yacyo.

Gushyira mu bikorwa Isi

Kuzamura ibizamini bya ResNet na DenseNet byerekana amashusho kuri CIFAR na ImageNet nta giciro cyinyongera.

SWAG (SWA-Gaussian) itanga igereranya ridashidikanywaho kubigereranyo byerekeranye n'umutekano biturutse kumyitozo imwe.

EMA-yuburemere ituma urusobe rwicyitegererezo mumashanyarazi atanga amashusho nka Diffusion Stable.

Kubaka 'isupu ntangarugero' ugereranije impuzandengo nyinshi zateguwe neza kugirango utezimbere imbaraga utarinze kwitoza.

Igihe kizaza cyo Kuringaniza Ibiro

SWA yabyaye ibintu bitandukanye nka SWA-Gaussian (SWAG) kubihendutse bya Bayesiya bihendutse, kandi igitekerezo cyo kugereranya ubu gishimangira amayeri ya Exponential Moving Average akoreshwa cyane muburyo bwo gukwirakwiza, kwigira wenyine, no kwigana urugero runini. Witege ko impuzandengo y'ibiro igumaho 'ifunguro rya sasita' isanzwe muburyo bwo guhugura, hamwe nubushakashatsi bugera no guhuza imiterere yigenga yigenga (isupu yicyitegererezo) no kunoza kalibrasi hamwe nukuri.

Kubishyira mubikorwa

Impamvu ari ngombwa

  • Igiciro na bije. Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
  • Ibyemezo bisobanutse. Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
  • Kugenzura ubuziranenge. Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.

Witondere

  • Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
  • Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
  • Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.

Intambwe ikurikira

  1. Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
  2. Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
  3. Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
  4. Tegura inzira yo gusubiza ibyabaye mbere yo gupima.

Komeza Ubushakashatsi

Ibaruwa y'amakuru y'ubuntu

Ibuka amakuru ya buri munsi ya AI

Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.

Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.

Isuzume

Take the Stochastic Weight Averaging quiz

Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.

Tangira ikibazo

Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano

Ibibazo bikunze kubazwa

What is Stochastic Weight Averaging?

Kugereranya Ibiro bya Stochastic (SWA) bifata impuzandengo yoroheje yuburemere bwikitegererezo kuva kumanota menshi yatinze mumyitozo aho gukomeza gufata ifoto yanyuma. Aya mayeri ahendutse akenshi yerekana icyitegererezo muburyo bushimishije, mugari mugace k'igihombo, gikunda kuba rusange muburyo bugaragara kumibare itagaragara.

Kugereranya Uburemere bwa Stochastique ni ikihe?

SWA igereranya ibipimo by'icyitegererezo (uburemere) byakusanyirijwe kuri bariyeri nyinshi mugice cyanyuma cyamahugurwa, ntabwo ari ibyahanuwe cyangwa gradients.

Kuki SWA ikunda kunoza rusange?

Impuzandengo yimura igisubizo cyerekeza hagati yakarere kegeranye k’ubutaka bwatakaye, kandi minima iringaniye muri rusange kuko gari ya moshi nigihombo cyibizamini bikomeza guhuza.

Ni izihe ntambwe yinyongera SWA isaba imiyoboro ikoresha ibice bisanzwe?

Kuberako imibare ya BN iterwa nuburemere bwihariye, impuzandengo yikigereranyo ikenera pass imwe yinyongera hejuru yamakuru kugirango ibaze uburyo bwo gukora nuburyo butandukanye.

Ni iyihe myitwarire yo kwiga-isanzwe ikoreshwa mugihe cyo kugereranya SWA?

SWA ikomeza igipimo giciriritse gihoraho cyangwa cyikigero cyo kwiga kuburyo SGD ikomeje gushakisha akarere kagari kangana amanota yabo agereranijwe.

Nigute ibiciro bya SWA byagereranijwe ugereranije na gakondo ya moderi ya N?

SWA isenya ibirindiro byinshi murwego rumwe rwagereranijwe, bityo rero umwanzuro ugura kimwe na moderi imwe aho kuba N.