Ci bi mu melGis bi ci topp
Empirical Risk Minimization
Fondamentaal yi
GUIDE teknik
Minimisation bu xam ñaw
Sharpness-Aware Minimization (SAM) xeetu gëna xéewale la bu bëggul rek ñàkk lu néew waaye ñàkk lu néew ci dëkkandoo yépp ci diisaay - lu gëna ndaw.
Déglul tegtal biiIA-nettali · ~2 min
Ci xët wii2 simili jàng
Résumé
Minima yu plat yi ñooy gëna mëna yamale, moo tax SAM dafay faral di gëna yombal njubte ak dëgëraay bi te du soppi architecture model bi.
Plongeur bu xóot
Standard training dafay wàññi perte ci benn poñ ci espace poids, waaye ñaari pexe yu am benn perte de training mën nañu doxalee ci anam wu wuute lool: benn 'sharp' minimum toog ci vale bu sew fu perturbations poids yu ndaw spike perturbation, ci noonu la 'flat' minimum tolerante perturbation te dafay faral di generalise done gëna un. SAM, bi gëstukat yi dugal ci 2020, dafay leeral lii. Ci jéego bu nekk, dafay njëkka gis perturbation poid bi gëna jege (ci biir radius rho bu ndaw) biy yokk perte bi - dëkkandoo bi gëna bon - ba noppi yeesal poid yi njëkk ngir wàññi perte bi ci point perturbed bi. Bii mébet min-max dafay puus optimisation ci gox yu gëna suufe, di joxe generalisation bu gëna baax ci wàllu nataal ak ginaaw.
Doxal ci àdduna dëgg
Yokkatal Transformatëru gis-gis ak ResNet ci ImageNet ci tàggat ak SAM ci barabu SGD bu leer.
Yokkateg dëgëraay ci bruit etiket yi, ndax minima yu plat yi ñoo gëna néew luñu mëna memorise etiket yu yàqu yi.
Fine-tuning xeetu làkk yiñ njëkka tàggat ak SAM ngir gëna am generalisation ci ensemble done yu ndaw yi ci suuf.
Jëfandikoo ESAM wala LookSAM sudee njëgu xayma bu ñaari yoon ci vanille SAM seer na lool.
Ëlëgu wàññig xam-xam bu ñaw
SAM jural na famiy topp-topp yu jublu ci ñakk kattan gu gëna mag, xayma ñaari yoon: anam yu am njariñ yu melni ESAM, LookSAM, ak pexe yuy yàq benn subset ci poid yi wala jëfandikoo SAM ci jéego yu néew yu nekk. SAM buy méngoo (ASAM) dafay soppi rayon bi mu baña soppiku ci eskaal bi. Gëstukat yi ñu ngi wéy di waxtaan ci li waral flatness di jàppale ak ni ñu koy nattee, ba noppi xalaat yu am ñaw ñu ngi tasaaroo ngir gëna suqali xeetu làkk yu mag yi ak gëna dëgëral coppite ci séddale bi.
Teg ko ci jëf
Lu tax mu am solo
- Njëgg ak budget. Dogal yi architecture di jël dañuy indi njariñ ak njëgu liggéey bi ay at ci ginaaw.
- dogal yu gëna leer. Njàngalem xarala yi dafay jàppale ekip yi ñu tànn li gën, te baña yam ci li gëna bees daal.
- Xool kalite. Tanneef yu gëna baax ci wàllu ingeñër dina wàññi jafe-jafe yi ci wàllu wóor ci liggéey bi.
Moytul
- Optimize benn benchmark mën na nëbb ñakk kattan yu gëna yaatu ci sistem bi.
- Njëg li ñuy fay ci infrastructure yi ak ci toppatoo dañuy faral di suufeel.
- Bu sistem yi di gëna xawa jafee xam, jafe-jafe yi am ci wàllu kaaraange ak seetlu mën nañu gëna bari.
Jéego yi ci topp
- Mandargal latency, kalite, ak njëg yi laata ngay jëfandikoo.
- Benchmark ci biir sargal ak done yu dëggu.
- Jumtukaay bi di saytu njuumte yi, derive bi ak njeextalu jëfandikukat bi.
- Waajal rollback ak yooni tontu ci jafe-jafe yi laata ngay eskale.
Weyal di banneexu
Bulletin bu amul fayda
Jàngal xibaar AI bu bes
Ñatti jaar-jaari IA yuñ firndeel bis bu nekk ci ayu-bis bi, ñu bind leen ci Àngle bu leer. Gratis ba fàww, amul yëgle.
Benn email bis bu nekk ci ayu-bis bi. Benn klik nga dindi abonemaa bi. Dunu musa jaay wala séddoo sa adres.
Nattal sa bopp
Take the Sharpness-Aware Minimization quiz
Feedback ci saasi ci bépp tontu, ak sertifikaa buñ mëna séddoo ak ID buñ mëna saytu soo jàllee ci njàng mi.
Jàppale njàngum IA te doo fay. AI Understanding mbootaay la buy def te Yàlla tax — amul yëgle, amul paywall, musul am. Defal ab sarax
Laaj yi ñuy faral di laaj
Luy wàññig xam-xam bu ñaw?
Sharpness-Aware Minimization (SAM) xeetu gëna xéewale la bu bëggul rek ñàkk lu néew waaye ñàkk lu néew ci dëkkandoo yépp ci diisaay - lu gëna ndaw. Minima yu plat yi ñooy gëna mëna yamale, moo tax SAM dafay faral di gëna yombal njubte ak dëgëraay bi te du soppi architecture model bi.
Ban xeetu minimum la SAM di jéema gis?
SAM dafay wut minima yu plat ndax perte bi des ci suufu perturbations poids yu ndaw yi dañuy gëna generalise ci done yiñ gisul.
Ñaata paas ci kanam ak ci ginaaw la jéego SAM buñ miin di soxla?
SAM dafay xayma benn gradient ngir gis point bi gëna bon ci jege, ba noppi beneen gradient foofu ngir yeesal - lu tollu ci ñaari yoon njëg bi ñuy faral di def.
Lan la hiperparametru rayon rho di saytu ci SAM?
Rho dafay wane fu jéego 'yéeg' bi di dem ngir gis dëkkandoo bi gëna bon, di fësal yaatuwaayu gox bu dalal bi SAM di wër.
Lan mooy li njëkk ci ñaari jéego yi SAM wara def ci iteration bu nekk?
SAM dafay njëkka perturber poid yi ci biir rayon rho ci yoon wi gëna rëy perte, ginaaw ga mu wàcci ci point bi njëkk jëfandikoo gradient biñ xayma foofu.
Lu tax SAM di faral di gëna dëgëral etiketu bruit yi?
Ngir mëna jàpp etiket yu bari bruit dafay laaj minima yu ñaw te sew; ci bëgg gox yu dalal, SAM baña nangu overfitting.
Weyal di jàng
Gid yu jëm ci loolu
Tann nañu yeneen njiit ngir topic bii