Matsakaicin Nauyin Stochastic
Matsakaicin Nauyin Stochastic (SWA) yana ɗaukar matsakaicin matsakaicin ma'aunin ƙirar daga maki da yawa a ƙarshen horo maimakon kawai adana hoton ƙarshe.
Dubawa
This cheap trick often lands the model in a flatter, wider region of the loss landscape, which tends to generalize noticeably better on unseen data.
Zurfafa nutsewa
Izmailov, Wilson da abokan aiki suka gabatar da su a cikin 2018, SWA ta yi amfani da lura cewa SGD tare da ƙimar koyo na akai-akai ko cyclical ba ya haɗuwa zuwa aya ɗaya - yana bounces a kusa da gefen wani fili mai faɗin kwari. Maimakon ɗaukar ɗayan waɗannan wuraren tsayawa masu hayaniya, SWA tana gudanar da matsakaicin matsakaicin matsakaici (sau da yawa akai-akai ko cyclic) ƙimar koyo don lokutan ƙarshe kuma yana daidaita ma'aunin nauyi da yake ziyarta, yawanci kowane zamani. Matsakaicin ma'aunin nauyi yana zama kusa da tsakiyar yankin faɗuwa. Saboda ana ƙididdige ƙididdiga na daidaitawa ga takamaiman ma'auni, SWA na buƙatar ƙarin wucewa ɗaya na gaba akan bayanai don sake ƙididdige hanyoyin tafiyar da BN da bambance-bambancen ƙima. Farashin ainihin kyauta ne, kuma daidaitattun ribar da aka samu sun yi daidai a cikin rarrabuwar hotuna da bayan haka.
Fahimtar Fasaha
SWA tana kiyaye matsakaicin gudu w_SWA = (n·w_SWA + w_i)/(n+1) ana sabunta kowane zagayowar, yayin da ƙirar SGD mai rai ke ci gaba da bincike tare da ƙimar koyo mai girma. Matsakaicin matsakaicin nauyi a sararin samaniya yana kusanta tari a sararin aiki amma yana biyan ƙira ɗaya bisa ƙididdigewa, ba da yawa ba. Makullin hanyar ita ce ƙaramin ƙaramin lebur yana da ƙarfi ga rikicewar nauyi, don haka filayen asarar horo/gwaji suna kasancewa a daidaita, rage tazarar gaba ɗaya.
Dabarun Tasiri
Kudin da kasafin kuɗi
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Shawarwari masu haske
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Kula da inganci
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Makomar Matsakaicin Nauyin Stochastic
SWA ta haifar da bambance-bambancen kamar SWA-Gaussian (SWAG) don rashin tabbas na Bayesian mai arha, kuma matsakaicin ra'ayin yanzu yana ƙarfafa Matsakaicin Matsakaicin Matsakaicin Dabaru da aka yi amfani da su sosai a cikin nau'ikan watsawa, koyo mai kulawa da kai, da manyan horon samfuri. Yi tsammanin matsakaicin nauyi ya kasance tsoho 'abincin rana kyauta' a cikin girke-girke na horo, tare da ƙaddamar da bincike don haɗa nau'ikan horarwa masu zaman kansu (miyan ƙira) da haɓaka daidaitawa tare da ɗanyen daidaito.
Aiwatar da Gaskiyar Duniya
Haɓaka daidaiton gwajin ResNet da DeseNet masu rarraba hoto akan CIFAR da ImageNet ba tare da ƙarin farashi ba.
SWAG (SWA-Gaussian) yana samar da ƙididdige ƙididdiga na rashin tabbas don tsinkaya-aminci daga aikin horo guda ɗaya.
EMA-na-nauyi yana daidaita hanyar sadarwar samfur a cikin masu samar da hoto kamar Stable Diffusion.
Gina 'miyan ƙira' ta hanyar maƙasudin madaidaitan wuraren bincike da yawa don inganta ƙarfi ba tare da sake horarwa ba.
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Stochastic Weight Averaging quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Fara nauyi
Tambayoyin da ake yawan yi
What is Stochastic Weight Averaging?
Matsakaicin Nauyin Stochastic (SWA) yana ɗaukar matsakaicin matsakaicin ma'aunin ƙirar daga maki da yawa a ƙarshen horo maimakon kawai adana hoton ƙarshe. Wannan dabarar mai arha sau da yawa tana saukar da ƙirar a cikin shimfidar wuri mai faɗi, yanki mafi fa'ida na yanayin asara, wanda ke ba da fifiko ga mafi kyawu akan bayanan gaibu.
Menene Matsakaicin Nauyin Stochastic a zahiri?
SWA tana daidaita ma'aunin ƙira (masu nauyi) waɗanda aka tattara a wuraren bincike da yawa yayin lokacin horo na ƙarshe, ba tsinkaya ko gradients ba.
Me yasa SWA ke son inganta gama gari?
Matsakaicin yana matsar da mafita zuwa tsakiyar yanki mai lebur na farfajiyar asara, kuma ƙaramar ƙasa ta fi dacewa saboda jirgin ƙasa da asarar gwajin sun kasance cikin layi.
Wane ƙarin mataki ne SWA ke buƙata don cibiyoyin sadarwar da ke amfani da daidaita tsari?
Saboda kididdigar BN ta dogara da takamaiman ma'auni, matsakaicin ƙira yana buƙatar ƙarin wucewa kan bayanai don sake ƙididdige hanyoyin gudana da bambance-bambance.
Wadanne halaye na koyo ne ake amfani da su a lokacin matsakaicin lokaci na SWA?
SWA tana kiyaye matsakaicin matsakaicin matsakaici ko ƙimar koyo na keke don haka SGD ta ci gaba da bincika faffadan yanki wanda aka kimanta maki.
Ta yaya farashin ƙaddamarwa na SWA ya kwatanta da tarin al'ada na ƙirar N?
SWA tana rushe wuraren bincike da yawa zuwa matsakaicin matsakaicin nauyi, don haka ƙimar ƙima iri ɗaya ce da ƙira ɗaya maimakon N.