Minimisation bu xam ñaw
Sharpness-Aware Minimization (SAM) xeetu gëna xéewale la bu bëggul rek ñàkk lu néew waaye ñàkk lu néew ci dëkkandoo yépp ci diisaay - lu gëna ndaw.
Résumé
Flatter minima tend to generalize better, so SAM often improves test accuracy and robustness without changing the model architecture.
Plongeur bu xóot
Standard training dafay wàññi perte ci benn poñ ci espace poids, waaye ñaari pexe yu am benn perte de training mën nañu doxalee ci anam wu wuute lool: benn 'sharp' minimum toog ci vale bu sew fu perturbations poids yu ndaw spike perturbation, ci noonu la 'flat' minimum tolerante perturbation te dafay faral di generalise done gëna un. SAM, bi gëstukat yi dugal ci 2020, dafay leeral lii. Ci jéego bu nekk, dafay njëkka gis perturbation poid bi gëna jege (ci biir radius rho bu ndaw) biy yokk perte bi - dëkkandoo bi gëna bon - ba noppi yeesal poid yi njëkk ngir wàññi perte bi ci point perturbed bi. Bii mébet min-max dafay puus optimisation ci gox yu gëna suufe, di joxe generalisation bu gëna baax ci wàllu nataal ak ginaaw.
Gis-gis xarala
Jéego bu nekk ci SAM ñaari yoon la. Bi njëkk mooy nga xayma degrade bi ci poid yi fi nekk, nga jël jéego 'yéeg' bu tollu ci rho ci wàllu degrade bi ngir yegg ci barab bi gëna bon ci wetu bi. Ñaareel ba mooy xayma degrade bi ci poñ perturbé bi nga jëfandikoo ko ngir yeesal poid yi njëkk. Rayon rho mooy wane yaatuwaayu dëkkandoo bi ngay aar ci. Njëgglangi tollu ci ñaari paas ci kanam-ci ginaaw ci jéego bu nekk, loolu mooy yokk ñaari yoon xayma bi - mooy gëna ñaawe ci jëfandikoo gi.
njeextalu pexe
Njëgg ak budget
Dogal yi architecture di jël dañuy indi njariñ ak njëgu liggéey bi ay at ci ginaaw.
dogal yu gëna leer
Njàngalem xarala yi dafay jàppale ekip yi ñu tànn li gën, te baña yam ci li gëna bees daal.
Xool kalite
Tanneef yu gëna baax ci wàllu ingeñër dina wàññi jafe-jafe yi ci wàllu wóor ci liggéey bi.
Ëlëgu wàññig xam-xam bu ñaw
SAM jural na famiy topp-topp yu jublu ci ñakk kattan gu gëna mag, xayma ñaari yoon: anam yu am njariñ yu melni ESAM, LookSAM, ak pexe yuy yàq benn subset ci poid yi wala jëfandikoo SAM ci jéego yu néew yu nekk. SAM buy méngoo (ASAM) dafay soppi rayon bi mu baña soppiku ci eskaal bi. Gëstukat yi ñu ngi wéy di waxtaan ci li waral flatness di jàppale ak ni ñu koy nattee, ba noppi xalaat yu am ñaw ñu ngi tasaaroo ngir gëna suqali xeetu làkk yu mag yi ak gëna dëgëral coppite ci séddale bi.
Doxal ci àdduna dëgg
Yokkatal Transformatëru gis-gis ak ResNet ci ImageNet ci tàggat ak SAM ci barabu SGD bu leer.
Yokkateg dëgëraay ci bruit etiket yi, ndax minima yu plat yi ñoo gëna néew luñu mëna memorise etiket yu yàqu yi.
Fine-tuning xeetu làkk yiñ njëkka tàggat ak SAM ngir gëna am generalisation ci ensemble done yu ndaw yi ci suuf.
Jëfandikoo ESAM wala LookSAM sudee njëgu xayma bu ñaari yoon ci vanille SAM seer na lool.
Risk yi ak balustrade yi
Optimize benn benchmark mën na nëbb ñakk kattan yu gëna yaatu ci sistem bi.
Njëg li ñuy fay ci infrastructure yi ak ci toppatoo dañuy faral di suufeel.
Bu sistem yi di gëna xawa jafee xam, jafe-jafe yi am ci wàllu kaaraange ak seetlu mën nañu gëna bari.
Roadmap ngir samp gi
Mandargal latency, kalite, ak njëg yi laata ngay jëfandikoo.
Benchmark ci biir sargal ak done yu dëggu.
Jumtukaay bi di saytu njuumte yi, derive bi ak njeextalu jëfandikukat bi.
Waajal rollback ak yooni tontu ci jafe-jafe yi laata ngay eskale.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Sharpness-Aware Minimization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
DenseNet ak lëkkaloo bu dëgër
Laaj yi ñuy faral di laaj
What is Sharpness-Aware Minimization?
Sharpness-Aware Minimization (SAM) xeetu gëna xéewale la bu bëggul rek ñàkk lu néew waaye ñàkk lu néew ci dëkkandoo yépp ci diisaay - lu gëna ndaw. Minima yu plat yi ñooy gëna mëna yamale, moo tax SAM dafay faral di gëna yombal njubte ak dëgëraay bi te du soppi architecture model bi.
Ban xeetu minimum la SAM di jéema gis?
SAM dafay wut minima yu plat ndax perte bi des ci suufu perturbations poids yu ndaw yi dañuy gëna generalise ci done yiñ gisul.
Ñaata paas ci kanam ak ci ginaaw la jéego SAM buñ miin di soxla?
SAM dafay xayma benn gradient ngir gis point bi gëna bon ci jege, ba noppi beneen gradient foofu ngir yeesal - lu tollu ci ñaari yoon njëg bi ñuy faral di def.
Lan la hiperparametru rayon rho di saytu ci SAM?
Rho dafay wane fu jéego 'yéeg' bi di dem ngir gis dëkkandoo bi gëna bon, di fësal yaatuwaayu gox bu dalal bi SAM di wër.
Lan mooy li njëkk ci ñaari jéego yi SAM wara def ci iteration bu nekk?
SAM dafay njëkka perturber poid yi ci biir rayon rho ci yoon wi gëna rëy perte, ginaaw ga mu wàcci ci point bi njëkk jëfandikoo gradient biñ xayma foofu.
Lu tax SAM di faral di gëna dëgëral etiketu bruit yi?
Ngir mëna jàpp etiket yu bari bruit dafay laaj minima yu ñaw te sew; ci bëgg gox yu dalal, SAM baña nangu overfitting.