Lalacewar Nauyi da Tsarin L2
Lalacewar nauyi wata hanya ce mai sauƙi, mai ƙarfi wacce ke karkatar da ma'aunin samfurin zuwa sifili yayin horo, yana hana shi dogaro da ƙarfi kan kowane fasali guda ɗaya.
Dubawa
It reduces overfitting and is one of the most widely used regularizers in deep learning.
Zurfafa nutsewa
Lokacin da samfurin ya yi horo, zai iya kama amo a cikin bayanai ta hanyar girma manyan ma'aunin nauyi masu kyau waɗanda suka dace da tsarin horon amma bai dace ba. L2 daidaitawa yana yaƙar wannan ta ƙara hukunci daidai da jimlar ma'aunin ma'auni zuwa aikin asara. Mai ingantawa yanzu yana da maƙasudai biyu: dacewa da bayanai kuma kiyaye ƙananan nauyi, don haka ya daidaita akan mafi santsi, mafi ƙarfi mafita. Rushewar nauyi shine ra'ayin da ke da alaƙa na rage kowane nauyi da ɗan ƙaramin juzu'i akan kowane matakin sabuntawa. Tare da zuriyar gradient a sarari su biyun sun yi daidai da lissafi, amma tare da masu inganta haɓakawa kamar Adam sun bambanta, wanda shine dalilin da ya sa aka gabatar da AdamW don lalata lalata daga sabuntawar tushen gradient kuma ya sanya shi yayi daidai.
Fahimtar Fasaha
Daidaita L2 yana ƙara lokutan lambda jimlar ma'aunin murabba'i ga asara, don haka gradient ɗinsa yana ƙara lokaci daidai da kowane nauyi, yana ja shi zuwa sifili. Rushewar nauyin nauyi a maimakon haka yana ninka kowane nauyi da ma'auni kamar (1 a debe learn_rate times lambda) kai tsaye. A cikin hanyoyin daidaitawa, haɗa L2 cikin asara yana ba da damar sikelin kowane siga ya karkatar da hukuncin, don haka AdamW yana amfani da raguwa daban, yana maido da ƙaƙƙarfan yunifom ɗin da aka yi niyya zuwa ƙananan ma'auni.
Dabarun Tasiri
Shawarwari masu haske
Yana taimaka muku keɓance bayyanannen da'awar fasaha daga harshen talla.
Kudin da kasafin kuɗi
Kuna iya yin mafi kyawun tambayoyin aiwatarwa kafin kashe kuɗi ko lokaci.
Ƙungiya da aikin aiki
Ƙungiyoyin da ke da fahimtar juna suna yin mafi kyawun samfura, manufofi, da yanke shawara na koyo.
Makomar Raba Nauyi da Tsarin L2
Lalacewar nauyi ya kasance babban sinadari na horar da girke-girke na manyan nau'ikan harshe da masu canza hangen nesa, kuma AdamW yanzu shine madaidaicin ingantawa a gare su. Bincike ya ci gaba da yin mu'amala da yadda lalata ke hulɗa tare da jadawalin koyo, daidaita matakan daidaitawa, da ma'aunin ƙira, tunda ingantaccen ƙarfinsa yana canzawa yayin da ƙira ke girma. Yi tsammanin ƙarin ƙa'ida, mai yuwuwar kowane-Layi ko tsararru-tsara-tsara kamar yadda binciken hyperparameter mai sarrafa kansa ya girma da nazarin doka.
Aiwatar da Gaskiyar Duniya
Ƙara nauyi_lalacewa a cikin PyTorch's AdamW ko SGD ingantawa lokacin horar da masu rarraba hoto don hana wuce gona da iri.
Daidaita madaidaicin lambda a cikin koma baya, ƙirar layin L2 na al'ada, don daidaita tsinkaya akan fasalulluka masu alaƙa.
Babban tsarin ƙirar harshe na horarwa da girke-girke waɗanda ke saita ƙaramin lalacewa (sau da yawa kusan 0.1) tare da jadawalin ƙimar koyo
Haɗa lalata nauyi tare da haɓaka bayanai da raguwa don kiyaye ƙaramin ƙirar likitanci daga haddace ƙayyadaddun sikanin horo.
Hatsari & Tsare-tsare
Ƙungiyoyi daban-daban na iya amfani da kalmar iri ɗaya daban, don haka ayyana iyaka da wuri.
Alamomi na iya yin kama da ƙarfi yayin da aikin zahirin duniya bai yi daidai ba.
Yin watsi da ingancin bayanai da tsare-tsaren kimantawa galibi yana haifar da sakamako mara ƙarfi.
Taswirar Hanya
Fara da ma'anar harshe a sarari na sakamakon da kuke buƙata.
Zaɓi ma'aunin nasara ɗaya da yanayin gazawa ɗaya kafin gwaji.
Gudun ƙaramin matukin jirgi tare da bayanan wakilci, ba saitin demo da aka goge ba.
Takardun inda Lalacewar Nauyi da Tsarin L2 ke taimakawa kuma inda hanyoyin mafi sauƙi suka fi kyau.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Weight Decay and L2 Regularization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Tsayawa
Tambayoyin da ake yawan yi
What is Weight Decay and L2 Regularization?
Lalacewar nauyi wata hanya ce mai sauƙi, mai ƙarfi wacce ke karkatar da ma'aunin samfurin zuwa sifili yayin horo, yana hana shi dogaro da ƙarfi kan kowane fasali guda ɗaya. Yana rage wuce gona da iri kuma yana ɗaya daga cikin abubuwan da ake amfani da su na yau da kullun a cikin zurfin koyo.
Menene daidaitawar L2 ya ƙara zuwa aikin asara?
Daidaita L2 yana ƙara lokutan lambda jimlar ma'auni murabba'i, ladabtar da manyan ma'aunin nauyi da ƙarfafa ƙarami, mafita mai santsi.
Menene babbar matsalar da lalata nauyi ke taimakawa hanawa?
Ta hanyar ɗora ƙananan ma'auni, lalata nauyi yana hana samfurin daga dacewa da amo, inganta haɓakawa zuwa sababbin bayanai.
A wace hanya ce lalacewar nauyi ke tura ma'aunin samfurin?
Raunin nauyi yana raguwa zuwa sifili akan kowane sabuntawa, wanda shine dalilin da ya sa a wasu lokuta ana kwatanta shi da 'lalata' ma'aunin nauyi.
Me yasa aka gabatar da AdamW?
A cikin Adamu, ninka L2 a cikin asarar yana mu'amala da mummuna tare da sikelin siga; AdamW yana shafa ruɓe daban don maido da raguwar yunifom da aka yi niyya.
Don zuriyar gradient a sarari, ta yaya daidaitawar L2 da lalata nauyi suke da alaƙa?
Tare da SGD bayyananne, ƙara hukuncin L2 ga asara yana haifar da sabuntawa iri ɗaya kamar ma'aunin raguwa kai tsaye, don haka biyun daidai suke.