Uhlolojikelele
I-Sharpness-Aware Minimization (SAM) iyindlela yokuthuthukisa engafuni nje ukulahlekelwa okuphansi kodwa ukulahlekelwa okuphansi kuyo yonke indawo yezisindo - ubuncane obuphansi. I-Flatter minima ijwayele ukujwayela kangcono, ngakho i-SAM ivamise ukuthuthukisa ukunemba kokuhlolwa nokuqina ngaphandle kokushintsha imodeli yezakhiwo.
I-Sharpness-Aware Minimization iyibhulokhi yokwakha yobuchwepheshe ethinta ikhwalithi yemodeli, izindleko zengqalasizinda, ukubambezeleka, nokuthembeka esikalini.
I-Deep Dive
Ukuqeqeshwa okujwayelekile kunciphisa ukulahlekelwa endaweni eyodwa esikhaleni sesisindo, kodwa izixazululo ezimbili ezinokuncipha okufanayo kokuqeqeshwa zingaziphatha ngendlela ehluke kakhulu: ubuncane 'obukhali' buhlala esigodini esincane lapho ukukhubazeka kwesisindo esincane kukhuphula ukulahlekelwa, kuyilapho ubuncane 'obucaba' bubekezelela ukuphazamiseka futhi ngokuvamile buhlanganisa kangcono idatha engabonakali. I-SAM, yethulwe Google abacwaningi ngo-2020, ikwenza kube sobala lokhu. Esinyathelweni ngasinye kuqala ithola ukunyakaziswa kwesisindo esiseduze (ngaphakathi kwerediyasi encane engu-rho) okwandisa ukulahlekelwa - umakhelwane osesimweni esibi kakhulu - bese ibuyekeza izisindo zangempela ukuze kwehliswe ukulahlekelwa kulelo phuzu eliphazamisekile. Le nhloso encane iphusha ukuthuthukiswa ezifundeni eziphansi ngokulinganayo, ezinikeza ukuvezwa okujwayelekile okungcono kakhulu ekuhlukaniseni izithombe nangale kwalokho.
I-Technical Insight
Isinyathelo ngasinye se-SAM singamaphasi amabili. Okokuqala, hlanganisa igradient ezisindweni zamanje bese uthatha isinyathelo 'sokukhuphuka' sikasayizi rho esiqondisweni segradient ukuze ufinyelele endaweni embi kakhulu eseduze. Okwesibili, hlanganisa i-gradient kulelo phuzu eliphazamisekile futhi ulisebenzise ukuze ubuyekeze izisindo zangempela. Irediyasi rho ilawula ukuthi inkulu kangakanani indawo ovikela kuyo. Izindleko cishe ziwukudlula okubili okuya phambili okuya emuva ngesinyathelo ngasinye, okuphinda kabili ukubala - okuwumphumela oyinhloko osebenzayo.
I-Mastering Sharpness-Aware Minimization
I-Sharpness-Aware Minimization (SAM) iyindlela yokuthuthukisa engafuni nje ukulahlekelwa okuphansi kodwa ukulahlekelwa okuphansi kuyo yonke indawo yezisindo - ubuncane obuphansi. I-Flatter minima ijwayele ukujwayela kangcono, ngakho i-SAM ivamise ukuthuthukisa ukunemba kokuhlolwa nokuqina ngaphandle kokushintsha imodeli yezakhiwo. I-Sharpness-Aware Minimization iyibhulokhi yokwakha yobuchwepheshe ethinta ikhwalithi yemodeli, izindleko zengqalasizinda, ukubambezeleka, nokuthembeka esikalini. Ukuze wakhe ukuqonda okujulile, phatha i-Sharpness-Aware Minimization njengemodeli yokusebenza, hhayi isici esisodwa: chaza imiphumela efiselekayo, ucacise ukucabanga, futhi uhlukanise lokho isistimu engakwenza ngokwethembeka kulokho okusadinga ukwahlulela kochwepheshe.
Empeleni, amaqembu aqinile asebenzisa i-Sharpness-Aware Minimization alungiselela izakhiwo, idatha, nokukhetha kwengqalasizinda ngokumelene nokuthembeka nezindleko. Babhala imibandela yempumelelo ecacile, ukuhlola okuqhathaniswa nedatha engokoqobo nokugeleza komsebenzi, futhi baphindaphinde ngokusekelwe kumaphethini okuhluleka aqashiwe esikhundleni sokuwina kwebhentshimakhi yesikhathi esisodwa. Yilapho ukuqonda kwethiyori kuguquka kube amandla ahlala njalo kuwo wonke umkhiqizo, inqubomgomo, kanye nokusebenza.
Izinqumo zezakhiwo ziqhuba ukusebenza kanye nezindleko zokusebenza iminyaka. Ngesikhathi esifanayo, Ukuthuthukisa ibhentshimakhi eyodwa kungafihla ubuthakathaka obubanzi besistimu. Indlela eqine kakhulu iwukuhlanganisa isivinini sokuhlola nesiyalo sokuphatha: qhuba abashayeli bezindiza, bamba ubufakazi, ushicilele amalogi ezinqumo, futhi ubuyekeze izivikelo ngokuqhubekayo njengoba imodeli yokuziphatha, okulindelwe ngabasebenzisi, kanye nezimfuneko zokulawula zishintsha.
I-Strategic Impact
Izinqumo zezakhiwo ziqhuba ukusebenza kanye nezindleko zokusebenza iminyaka.
Izinqumo zezakhiwo ziqhuba ukusebenza kanye nezindleko zokusebenza iminyaka. Ekusetshenzisweni kwekhwalithi ephezulu, lokhu kuhunyushwa emithethweni yokusebenza elinganisekayo, imingcele yobunikazi, nemikhuba yokubuyekeza ephindelelayo ukuze amaqembu akwazi ukukala ukuzethemba esikhundleni sokukala ukungaqondakali.
Imfundo yobuchwepheshe isiza amaqembu ukuthi akhethe isitaki esifanele, hhayi nje esisha.
Imfundo yobuchwepheshe isiza amaqembu ukuthi akhethe isitaki esifanele, hhayi nje esisha. Ekusetshenzisweni kwekhwalithi ephezulu, lokhu kuhunyushwa emithethweni yokusebenza elinganisekayo, imingcele yobunikazi, nemikhuba yokubuyekeza ephindelelayo ukuze amaqembu akwazi ukukala ukuzethemba esikhundleni sokukala ukungaqondakali.
Izinketho ezingcono zobunjiniyela zinciphisa izehlakalo ezinokwethenjelwa ekukhiqizeni.
Izinketho ezingcono zobunjiniyela zinciphisa izehlakalo ezinokwethenjelwa ekukhiqizeni. Ekusetshenzisweni kwekhwalithi ephezulu, lokhu kuhunyushwa emithethweni yokusebenza elinganisekayo, imingcele yobunikazi, nemikhuba yokubuyekeza ephindelelayo ukuze amaqembu akwazi ukukala ukuzethemba esikhundleni sokukala ukungaqondakali.
Ukuqaliswa Komhlaba Wangempela
Ukuthuthukisa I-Vision Transformer kanye nokunemba kwe-ResNet ku-ImageNet ngokuqeqeshwa ne-SAM esikhundleni se-SGD esobala.
Ukuthuthukisa ukuqina kokulebula umsindo, njengoba i-flat minima mancane amathuba okuba ibambe ngekhanda amalebula onakele.
Ukushuna kahle amamodeli olimi aqeqeshwe kusengaphambili nge-SAM ukuze uthole ukujwayezwa kangcono kumadathasethi amancane omfula.
Ukusebenzisa okuhlukile kwe-ESAM noma kwe-LookSAM lapho izindleko zekhompyutha eziphindwe kabili ze-vanilla SAM zibiza kakhulu.
Amaphethini Okusebenzisa
Ukunciphisa Ukuqaphela Ubukhali ekusebenzeni
Ukuthuthukisa I-Vision Transformer kanye nokunemba kwe-ResNet ku-ImageNet ngokuqeqeshwa ne-SAM esikhundleni se-SGD esobala.
Ukuthuthukisa I-Vision Transformer kanye nokunemba kwe-ResNet ku-ImageNet ngokuqeqeshwa ne-SAM esikhundleni samaQembu e-SGD asobala ngokuvamile athola imiphumela engcono lapho echaza imingcele yekhwalithi ngaphambili, egcina indlela yokukhuphuka komuntu yamakesi asemaphethelweni, futhi alandelele kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
Ukunciphisa Ukuqaphela Ubukhali ekusebenzeni
Ukuthuthukisa ukuqina kokulebula umsindo, njengoba i-flat minima mancane amathuba okuba ibambe ngekhanda amalebula onakele.
Ukuthuthukisa ukuqina kokulebula umsindo, njengoba i-flat minima mancane amathuba okuba ibambe ngekhanda amalebula onakele Amaqembu ngokuvamile athola imiphumela engcono lapho echaza izinga eliphezulu ngaphambili, egcina indlela yokukhuphuka komuntu yamakesi asemaphethelweni, futhi alandelele kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
Ukunciphisa Ukuqaphela Ubukhali ekusebenzeni
Ukushuna kahle amamodeli olimi aqeqeshwe kusengaphambili nge-SAM ukuze uthole ukujwayezwa kangcono kumadathasethi amancane omfula.
Ukushuna kahle amamodeli olimi aqeqeshwe kusengaphambili nge-SAM ukuze ajwayezwe kangcono kumadathasethi amancane ahamba phansi Amaqembu ngokuvamile athola imiphumela engcono uma echaza izinga eliphezulu ngaphambili, egcina indlela yokukhuphuka yabantu yamakesi asemaphethelweni, futhi alandelela kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
Ukunciphisa Ukuqaphela Ubukhali ekusebenzeni
Ukusebenzisa okuhlukile kwe-ESAM noma kwe-LookSAM lapho izindleko zekhompyutha eziphindwe kabili ze-vanilla SAM zibiza kakhulu.
Ukusebenzisa okuhlukile kwe-ESAM noma kwe-LookSAM lapho izindleko zekhompiyutha eziphindwe kabili ze-vanilla SAM zibiza kakhulu Amathimba ngokuvamile athola imiphumela engcono uma echaza izinga eliphezulu ngaphambili, egcina indlela yokukhuphuka yomuntu yamakesi asemaphethelweni, futhi alandelele kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
Izingozi & Guardrails
Ukuthuthukisa ibhentshimakhi eyodwa kungafihla ubuthakathaka obubanzi besistimu.
Izindleko zengqalasizinda nezokulungisa zivame ukubukelwa phansi.
Izikhala zokuphepha nokubonakala zingakhula njengoba izinhlelo ziba nzima kakhulu.
Ukuqalisa Umhlahlandlela
Chaza ukubambezeleka, ikhwalithi, nezindleko ezihlosiwe ngaphambi kokuqaliswa.
Chaza ukubambezeleka, ikhwalithi, nezindleko ezihlosiwe ngaphambi kokuqaliswa. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.
Ibhentshimakhi ngaphansi komthwalo wangempela nezimo zedatha.
Ibhentshimakhi ngaphansi komthwalo wangempela nezimo zedatha. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.
Ukuqapha amathuluzi amaphutha, ukukhukhuleka, nomthelela wabasebenzisi.
Ukuqapha amathuluzi amaphutha, ukukhukhuleka, nomthelela wabasebenzisi. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.
Lungiselela izindlela zokuhlehlisa nezigameko ngaphambi kokukala.
Lungiselela izindlela zokuhlehlisa nezigameko ngaphambi kokukala. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.