Lottery Tikiti Hypothesis
Iyo Lottery Tikiti Hypothesis inotaura kuti mukati mehombe, yakatangwa neural network inovanza diki diki - 'tiketi rekuhwina' - iro, rakadzidziswa rega kubva kune mamwe maremu ekutanga, rinogona kuenderana netiweki yakazara nekurongeka.
Pfupiso
It matters because it suggests we are training far more parameters than we actually need.
Kudzika Kwakadzika
Yakakurudzirwa naJonathan Frankle naMichael Carbin kuMIT muna 2018, iyo hypothesis yakakura kubva mukuchekerera tsvagiridzo. Kazhinji iwe unogona kuchekerera network yakadzidziswa kudzika kusvika 10-20% yehuremu hwayo usingarasikirwe nemazvo, asi kudzidzisa iyo diki network kubva kutanga inotadza. Frankle naCarbin vakawana chikiribidi: chengetedza hukama huripo huremu hwekutanga. Iyo sparse subnetwork - tikiti rekuhwina - rinobva radzidzira kuzara chaiko mukuzviparadzanisa nevamwe, dzimwe nguva nekukurumidza kupfuura dense yepakutanga. Vakaona matikiti kuburikidza ne 'iterative magnitude pruning': chitima, chekerera huremu hudiki-hukuru, dzosera mamwe asara kune yawo yekutanga, uye dzokorora. Mhedzisiro yacho inoreva kuti dense overparameterization inonyanya kubatsira optimization kuwana yakanaka sparse chimiro, kwete kuti ese mauremu anodiwa ega.
Technical Insight
Iyo yakakosha maitiro ndeye iterative ukuru kuchekerera nekudzoreredza huremu: mushure mekudzidziswa, bvisa iyo yakaderera-ukuru huremu, dzorera maremu asara kune iwo ekutanga kutanga (kana yekutanga-yekudzidzira yekutarisa, kunatsa kunonzi 'kudzosera kumashure'), wozodzidzirazve. Iko kusanganiswa kweiyo sparse mask UYE kuenzanirana kwayo kwekutanga ndiko kunoita kuti tikiti 'rihwine' - kudzokorora masiki imwechete kunoparadza mhedzisiro.
Strategic Impact
Sarudzo dzakajeka
Inokubatsira kuparadzanisa zvakajeka zvichemo zvehunyanzvi kubva mumutauro wekushambadzira.
Mutengo uye bhajeti
Iwe unogona kubvunza zvirinani kuita mibvunzo usati washandisa mari kana nguva.
Team uye workflow
Zvikwata zvine nzwisiso yakagovaniswa inoita zvirinani chigadzirwa, mutemo, uye sarudzo dzekudzidza.
Ramangwana reLottery Tikiti Hypothesis
Lottery matikiti emafuta ekutsvagisa mukudzidzisa sparse network kubva pakutanga kuchengetedza komputa uye simba, uye mukati mekuti matikiti anotamisa mumadhata uye mabasa. Kuwedzera kuchekerera mabhirioni-parameter modhi kunoramba kuchidhura, saka basa rinoenderera nekutsvaga matikiti zvakachipa kana kuratidza kuti ariko (iyo 'yakasimba' yerotari yetiketi hypothesis inoti matikiti aripo pakutanga asina kudzidziswa zvachose). Tarisira ma-tie-ins ane anoshanda pa-mudziyo modhi uye yakasvibira AI.
Real-World Implementation
Kudzvanya kirasi yemufananidzo muhombe kusvika pasi pe20% yehuremu hwayo hwekutumirwa pafoni uchichengeta chokwadi.
Kumhanyisa kudzidziswa nekuziva uye kudzidzisa chete mashoma anokunda subnetwork
Kudzidza kuchinjisa uremu nekushandisazve tikiti rinowanikwa pane imwe dhatabheti kusvetuka-kutanga kudzidziswa pane yakabatana
Kuderedza simba rekufungidzira uye ndangariro mumidziyo yemupendero nekutumira tikiti rekuhwina rakatemwa panzvimbo peiyo dense modhi.
Njodzi & Guardrails
Zvikwata zvakasiyana zvinogona kushandisa izwi rimwechete zvakasiyana, saka tsanangura nzvimbo nekukurumidza.
Benchmarks inogona kutaridzika yakasimba nepo chaiyo-yenyika kuita isina kuenzana.
Kuregeredza mhando yedata uye zvirongwa zvekuongorora zvinowanzogadzira mhedzisiro isina kusimba.
Implementation Roadmap
Tanga netsanangudzo yemutauro wakajeka yemhedzisiro yaunoda.
Sarudza metric imwe yekubudirira uye imwe yekutadza mamiriro usati waedzwa.
Mhanya mutyairi mudiki ane data remumiriri, kwete demo rakakwenenzverwa.
Chinyorwa uko Lottery Tikiti Hypothesis inobatsira uye uko nzira dzakareruka dziri nani.
Ramba Uchiongorora
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Lottery Ticket Hypothesis quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gaidhi rinotevera
Chinchilla Compute-Optimal Training
Mibvunzo inowanzo bvunzwa
What is Lottery Ticket Hypothesis?
Iyo Lottery Tikiti Hypothesis inotaura kuti mukati mehombe, yakatangwa neural network inovanza diki diki - 'tiketi rekuhwina' - iro, rakadzidziswa rega kubva kune mamwe maremu ekutanga, rinogona kuenderana netiweki yakazara nekurongeka. Izvo zvine basa nekuti zvinoratidza kuti tiri kudzidzisa maparamita akawanda kupfuura atinoda chaizvo.
Chii chinonzi 'hwina tikiti' mune iyi hypothesis?
Tikiti rekuhwina idiki diki network iyo, kana yakadzidziswa yakazvimiririra kubva kune imwechete uremu hwekutanga, inosvika pakurongeka kunofananidzwa nedense network.
Sei kudzidzisa iyo yakatemwa subnetwork ichiwanzo kutadza kunze kwekunge iwe waita chimwe chinhu chakakosha?
Muono wakakosha ndewekuti sparse mask inoshanda chete kana yapetwa neyakafanana yekutanga yekutanga; random reinitialization inoityora.
Ndiani akakurudzira iyo Lottery Tikiti Hypothesis?
Jonathan Frankle naMichael Carbin vakaunza iyo hypothesis mune yavo 2018 MIT bepa.
Ndeipi nzira inoshandiswa kutsvaga matikiti ekuhwina?
Kuchekerera ukuru kunodzokororwa kunodzidzisa kakawanda, kunobvisa huremu hudiki-diki, uye kunodzosera mamwe asara kuhunhu hwawo hwekutanga.
Ko iyo hypothesis inoratidzei nezve yakakura overparameterized network?
Kuwana kunoreva overparameterization inobatsira kutsvaga kweinodzidziswa sparse subnetwork pane kuti huremu hwese hunodiwa.