Ilebula ethi Smoothing
Ukwenza ilebula bushelelezi iqhinga elilula lokujwayela elithambisa izinhloso zokuqeqesha ezishisayo, ukutshela imodeli impendulo efanele kungenzeka kakhulu kodwa akuqiniseki ngamaphesenti ayi-100.
Uhlolojikelele
It improves calibration and generalization across image and language models with almost no extra cost.
I-Deep Dive
Ngokuvamile umuntu ohlukanisa isigaba uqeqeshelwa amalebula ashisayo: ikilasi langempela lithola ithagethi engu-1.0 nayo yonke enye into engu-0.0. Kuhlanganiswe ne-cross-entropy kanye ne-softmax, lokhu kucindezela imodeli ukuze yenze ilogi elungile ibe mkhulu ngokungenamkhawulo kunayo yonke enye, okukhuthaza ukuzethemba ngokweqile kanye nokugcwalisa ngokweqile. Ukushelela kwelebula kungena esikhundleni sethagethi ngokuthi (1 - epsilon) yekilasi langempela kanye ne-epsilon/(K-1) esabalale kwamanye amakilasi angu-K, lapho i-epsilon incane (ngokuvamile engu-0.1). Imodeli manje ihlose ukusabalalisa okuzethembayo-kodwa-hhayi-absolute. Yethulwe emsebenzini ka-2016 we-Inception-v3 futhi kamuva yahlaziywa iqembu lika-Hinton, yathuthukisa ukunemba kwe-ImageNet futhi isezingeni kuma-Transformers, lapho iphepha lokuqala elithi Attention Is All You Need lisebenzise i-epsilon engu-0.1.
I-Technical Insight
Ngamalebula aqinile, ukunciphisa i-cross-entropy kushayela ilogu elilungile libheke ku-infinity enhle ngokuhlobene nabanye, okuyinto engenakufezwa futhi iphusha izisindo zibe ngokweqisa. Ukushelela kusetha igebe elilungile elilinganiselwe phakathi kwelogithi elungile nokunye, ngakho amalogi ahlala elinganiselwe futhi imodeli iyeka ukuzethemba okukhulu. Ucwaningo lubonisa lokhu kuqinisa amaqoqo ekilasi elifanayo futhi kukhiqiza amathuba alinganiswa kangcono, ukuzethemba okubikezelwe kufana nokunemba kwangempela. I-tradeoff: ingasula ulwazi oluqinile phakathi kwekilasi lokufana, ngezinye izikhathi okulimaza ukuchithwa kolwazi lapho lobo budlelwano obuthambile bubalulekile.
I-Strategic Impact
Izinqumo ezicacile
Kukusiza ukuthi uhlukanise izimangalo ezicacile zobuchwepheshe kusukela olimini lokumaketha.
Izindleko kanye nesabelomali
Ungabuza imibuzo yokusebenzisa kangcono ngaphambi kokusebenzisa imali noma isikhathi.
Ithimba kanye nokusebenza komsebenzi
Amaqembu anokuqonda okwabiwe enza izinqumo ezingcono zomkhiqizo, inqubomgomo, nokufunda.
Ikusasa Lelebula Ukushelela
Ukushelela kwelebula kuhlala kuwukuzenzakalelayo ekuqeqeshweni kwezinga elikhulu, kodwa ucwaningo luqhubekela phambili ekuguquleni nasekufundeni ukushelela okulungisa i-epsilon ngesibonelo ngasinye noma ngekilasi kunokusebenzisa inani eliyisicaba elilodwa. Izindlela ezigxile ekulinganisweni ezifana nokulahlekelwa okugxilwe kukho kanye nokukalwa kwezinga lokushisa ngokuvamile kukalwa ngakho noma kuhlanganiswe nakho. Njengoba amamodeli akhula futhi izilinganiso ezinokwethenjelwa zokungaqiniseki ziba bucayi kakhulu kwezokuphepha, lindela ukushelela kube ithuluzi elilodwa phakathi kwamaningi lokukhiqiza amaphuzu okuthenjwa athembekile, ngokunaka ngokucophelela ukungqubuzana kwawo okwaziwayo ne-distillation.
Ukuqaliswa Komhlaba Wangempela
Ukuhlukaniswa kwe-ImageNet: I-Inception-v3 isebenzise ilebula eshelelayo (epsilon 0.1) ukuze kukhuliswe ukunemba okuphezulu koku-1 nokunciphisa ukuzethemba ngokweqile.
Ukuhumusha ngomshini: i-Transformer yoqobo isebenzise ilebula yokushelela kuka-0.1, ihweba ngokudideka okuncane ukuze uthole amaphuzu aphezulu e-BLEU.
Ukuqashelwa kwenkulumo: okuqondiwe okushelelezi kunciphisa ukungaboni kahle kokuzethemba ngokweqile futhi kuthuthukise ukulinganiswa komsindo onomsindo.
Amamodeli ezithombe zezokwelapha: ukwenza bushelelezi kuveza amathuba alinganiswe kangcono, okubalulekile lapho amaphuzu okuzethemba azisa izinqumo zomtholampilo.
Izingozi & Guardrails
Amaqembu ahlukene angasebenzisa igama elifanayo ngokuhlukile, ngakho chaza ububanzi kusenesikhathi.
Amabhentshimakhi angabukeka eqinile kuyilapho ukusebenza komhlaba wangempela kungalingani.
Ukuziba ikhwalithi yedatha nezinhlelo zokuhlaziya kuvame ukudala imiphumela entekenteke.
Ukuqalisa Umhlahlandlela
Qala ngencazelo yolimi olulula yomphumela oyidingayo.
Khetha imethrikhi eyodwa yempumelelo nesimo esisodwa sokuhluleka ngaphambi kokuhlolwa.
Qalisa umshayeli omncane onedatha emele, hhayi isethi yedemo ephucuziwe.
Bhala lapho i-Label Smoothing isiza khona nalapho izindlela ezilula zingcono khona.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Label Smoothing quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Umhlahlandlela olandelayo
K-Means Clustering
Imibuzo evame ukubuzwa
What is Label Smoothing?
Ukwenza ilebula bushelelezi iqhinga elilula lokujwayela elithambisa izinhloso zokuqeqesha ezishisayo, ukutshela imodeli impendulo efanele kungenzeka kakhulu kodwa akuqiniseki ngamaphesenti ayi-100. Ithuthukisa ukulinganisa kanye nokwenza okuvamile kuwo wonke amamodeli wesithombe nezilimi cishe ngaphandle kwezindleko ezengeziwe.
Ngabe ukushelela kwelebula kushintsha ngani mayelana nezinhloso zokuqeqesha?
Esikhundleni sethagethi eqinile engu-1.0/0.0, ukushelela ilebula kwabela (1 - epsilon) ekilasini langempela futhi kusabalalise i-epsilon kwamanye.
Iyiphi inkinga ngamalebula aqinile okushisa okukodwa okulungisayo?
Ukunciphisa i-cross-entropy ekuqondisweni okushisayo kushayela ukungena okuqondile ku-infinity ngokuhlobene nabanye, okukhuthaza ukuzethemba ngokweqile kanye nokugcwalisa ngokweqile.
Yiziphi izakhiwo eziyingqopha-mlando ezisize ekwenzeni amalebula ashelelezi?
I-label smoothing yethulwe ephepheni le-Inception-v3 lango-2016 futhi yamukelwa yi-Transformer (Ukunakwa Yikho Konke Okudingayo) nge-epsilon 0.1.
Ngaphandle kokunemba, iyiphi inzuzo eyaziwa ngayo ukushelela ilebula?
Ukushelela kuthuthukisa ukulinganiswa, ukuzethemba okubikezelwe kuqondaniswe eduze namathuba eqiniso okulungile.
Iyini i-downside ebhaliwe yokushelela kwelebula?
Ngokuqinisa amaqoqo kanye nokwenza isicaba ubudlelwano obuthambile phakathi kwamakilasi, ukwenza bushelelezi kungasusa ulwazi ukukhishwa kwe-distillation kuncike kulo.