Decay de poids et régulation L2
Weight decay pexem bu yomb la, am doole buy gëna xëcc poids model bi ci zero ci diiru tàggat yaram, diko tere yéem lool ci benn manndarga.
Résumé
It reduces overfitting and is one of the most widely used regularizers in deep learning.
Plongeur bu xóot
Su benn model di tàggatoo, mën na tëye ci bruit bi ci done yi ci màgg, diisaay yu yaatu, yu méngoo bu baax ak tàggat yaram bi waaye du generalise bu baax. L2 regularisation dafay xeex loolu ci yokk penalti bu méngoo ak limu poids carré yi ci fonction perte bi. Optimizer bi leegi amna ñaari mébet: méngale done yi ak tëye poid yi, suko defee mu mëna tàmbali ci pexe yu gëna nooy, gëna dëgër. Weight decay mooy xalaat bi gëna jege di wàññi poids bu nekk ci fraction bu ndaw ci jéego bu nekk ci yeesal. Ak wàccinu gradient bu leer, ñaar ñi dañuy nuru ci wàllu math, waaye ak optimisatëri adaptif yu melni Adam dañu wuute, moo tax ñu dugal AdamW ngir dindi yàqu-yàqu ci yeesali gradient bi, def ko mu doxalee ci anam wu jaar yoon.
Gis-gis xarala
L2 regularisation dafay yokk lambda yoon limu poids carré yi ci perte bi, kon gradient bi dafay yokk terme bu méngoo ak poids bu nekk, diko yóbbu ci zero. Decouple poids decay lu moy loolu dafay yokk poid bu nekk ak facteur bu melni (1 dindi taux_learning yoon lambda) ci saasi. Ci anam yi ñuy méngoo, boole L2 ci ñàkk bi dafay tax eskaalu parametre bu nekk di soppi penalti bi, moo tax AdamW dafay jëfandikoo shrinkage bi ci boppam, defaraat pull uniform biñ bëggoon ci poid yu gëna ndaw.
njeextalu pexe
dogal yu gëna leer
Daf lay jàppale nga tàqale kàddu yu leer ci wàllu xarala ak làkku fësal njaay.
Njëgg ak budget
Mën nga laaj laaj yu gëna baax ci samp gi balaa ngay dugal xaalis wala sa jotu liggéey.
Ekip ak def liggéey
Ekip yi bokk xam-xam ñoo gëna mëna jël yenn dogal ci wàllu produit, politik ak jàng.
Ëlëgu ñakk yaram ak yamale L2
Poids decay des ingredient default ci recettes de formation ngir model lakk yu mag ak transformateur vision, te AdamW mooy leegi optimiser standard ci ñoom. Gëstu baa ngi wéy ci ni yàqu-yàqu di doxee ak jamonoy jàng, diisaayu normalisasioŋ, ak eskaalu model, ndax dooleem dafay soppeeku lu model yi di màgg. Xaarandil lu gëna am solo, amaana ci couche bu nekk wala ci jamonoy xam-xam ci seetug hyperparametre otomatik ak njàngum yoon ci escalier.
Doxal ci àdduna dëgg
Yokk weight_decay ci AdamW wala SGD bu PyTorch sooy tàggat ay nataal ngir wàññi limu ëpp
Defar koeffisientu lambda ci régresioŋ ridge, xeetu ligneer buñ daan L2 bu yàgg bi, ngir dakkal waxtaan yi ci màndarga yi korrele
Lakk bu yaatu modelu rëset bu njëkk def ab diisaay bu ndaw (dafay faral di nekk ci diggante 0.1) ci wetu ab kalendriye jàng-taux
boole diisaay bi ak yokk ay done ak bàyyi njàng ngir moytu ab modelu nataalu medsin bu ndaw di xam scanner yu néew ci tàggat yaram
Risk yi ak balustrade yi
Ekip yu bari mën nañu jëfandikoo benn baat ci anam wu wuute, kon teela leeral yaatuwaayam.
Benchmark yi mën nañu nuru lu am doole waaye performance yi ci àdduna bi duñu tolloo.
Bëgg kalite done ak palaŋu jàngat dafay faral di jur njariñ yu yomba dagg.
Roadmap ngir samp gi
Tàmbaleel ci joxe leeral ci làkk wu leer ci njariñ li nga soxla.
Tannal benn metric bu baax ak benn anam bu baaxul balaa ngay saytu.
Doxal ab pilote bu ndaw ak ay done yu representatif, du ab demo bu leer.
Bindal fi Weight Decay ak L2 Regularization di jàppale ak fi pexe yu gëna yomba gëna baax.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Weight Decay and L2 Regularization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Yamale
Laaj yi ñuy faral di laaj
What is Weight Decay and L2 Regularization?
Weight decay pexem bu yomb la, am doole buy gëna xëcc poids model bi ci zero ci diiru tàggat yaram, diko tere yéem lool ci benn manndarga. Dafay wàññi overfitting te nekkna benn ci regularizers yiñ gëna jëfandikoo ci jàng bu xóot.
Lan la L2 regularisation yokk ci fonction perte bi?
L2 regularisation dafay yokk lambda yoon limu poids carré yi, di yar poids yu mag yi, di ñaax pexe yu gëna ndaw, yu gëna nooy.
Lan mooy jafe-jafe bi gëna mag bi yaram mëna moytu?
Suñu tëyee poids yu ndaw, poids decay dafay tere model bi mën ànd ak bruit, gëna yombal generalisation ci done yu bees.
Ci ban wàll la diisaay biy dakkal di puus diisaayu model bi?
Weight decay dafay wàññi poids yi ci zero ci yeesal bu nekk, moo tax yenn saa yi dañu koy woowe 'decay' poid yi.
Lu tax ñu dugal AdamW?
Ci Adam, plier L2 ci perte bi defay jaxasoo bu baax ak scaling parametre bu nekk; AdamW dafay jëfandikoo decay bu wuute ngir defaraat shrinkage biñ bëggoon.
Ngir wàccinu gradient stochastic bu leer, naka la L2 di doxalee ak diisaay biy wàññeeku?
Ak SGD bu leer, yokk benn L2 penalti ci ñàkk bi dafay defar benn yeesal bu directement di wàññi poid yi, kon ñaar ñi ñooy benn.