Hipothèse biye loteri
Ticket Lottery Hypothesis dafa wax ni ci biir benn reso neuronal bu mag, buñ tàmbali ci anam wu mujjee, dafa nëbb benn subnetwork bu ndaw - 'ticket buy jël ndam' - bi, tàggat boppam ci benn poids bi njëkk, mën méngoo ak njubteg reso bi yépp.
Résumé
Dafa am solo ndax dafay wane ni danuy tàggat ay parametre yu ëpp li ñu soxla.
Plongeur bu xóot
Jonathan Frankle ak Michael Carbin ñoo ko taxawal ci MIT ci 2018, xalaat bi dafa juddoo ci gëstub dagg. Daanaka mën nga dagg reso buñ tàggat ba 10-20% ci diisaayam te doo ñàkk njub, waaye tàggat reso bu ndaw boobu ci noonu rek du am njariñ. Frankle ak Carbin gis nañu pexe bi: denc poids bi njëkk ci lëkkaloo yi des. Boobu subnetwork bu néew - ticket bi jël ndam li - dafay tàggat ci njubte bu mat sëkk ci isolation, yenn saa mu gëna gaaw ci dense original bi. Ñu ngi xàmmee këyit yi ci 'dagg magnitude iteratif': tàggat, dagg poids yu magnitude bi gëna ndaw, delloosi yeneen yi ci seen valeur yu njëkk, ba noppi baamtu. Resultaa bi dafay tekki ni overparameterization bu dëgër bi dafay jàppale optimisation bi mu am structure bu baax, te du ni poid yooyu yépp dañu soxla benn-benn.
Gis-gis xarala
Doxalin bi gëna am solo mooy dagg magnitude iteratif ak rebobinage poid: ginaaw tàggat, dindi poids yu magnitude bi gëna ndaw, reset poid yi des ci seen initialisation bu njëkk (wala checkpoint bu teel tàggat, ab setal bu ñuy woowe 'rebobinage'), ba noppi tàggataat. Njaxasu benn mask bu bariwul ak initialisation bi mu méngoo mooy tax ticket 'gañe' - initialisation bu bees ci benn mask bi dafay yàq effet bi.
njeextalu pexe
dogal yu gëna leer
Daf lay jàppale nga tàqale kàddu yu leer ci wàllu xarala ak làkku fësal njaay.
Njëgg ak budget
Mën nga laaj laaj yu gëna baax ci samp gi balaa ngay dugal xaalis wala sa jotu liggéey.
Ekip ak def liggéey
Ekip yi bokk xam-xam ñoo gëna mëna jël yenn dogal ci wàllu produit, politik ak jàng.
Ëlëgu xalaatu biye loteri
Ticket loteri fuel gëstu ci tàggat reso sparse ci ndoorte li ngir sakkanal ordinatër ak energie, ak ci ndax ticket yi dañuy toxal ci done yi ak liggéey yi. Scaling iterative pruning ba ci model yu am ay miliyaar ciy parametre mingi wéy di seer, kon liggéey bi dafay wéy ci am biye yu yomb wala ñu firndeel ni am nañu (hypothèse biye loteri 'bu dëgër' bi dafa wax ni biye yi am nañu ci ndoorte li te amul benn tàggat). Xaarandi lëkkaloo ak model aparey yu am doole ak IA bu wert.
Doxal ci àdduna dëgg
Komprime ab classifier nataal bu rëy ba ci suufu 20% ci diisaayam ngir dugal ko ci ab telefon boole ci tëye njub
Gaawaale tàggat ci ràññee ak tàggat rek benn sous-réseau buy am ndam
Jàngat toxal diisaay bi ci jëfandikoowaat ab këyit buñ fekk ci benn done ngir tàmbali tàggat yaram ci beneen bu jëm ci
Wàññi energie inference ak mémoire ci aparey edge yi ci yónnee këyit biy jël ndam li ci palaasu model bu dëgër bi
Risk yi ak balustrade yi
Ekip yu bari mën nañu jëfandikoo benn baat ci anam wu wuute, kon teela leeral yaatuwaayam.
Benchmark yi mën nañu nuru lu am doole waaye performance yi ci àdduna bi duñu tolloo.
Bëgg kalite done ak palaŋu jàngat dafay faral di jur njariñ yu yomba dagg.
Roadmap ngir samp gi
Tàmbaleel ci joxe leeral ci làkk wu leer ci njariñ li nga soxla.
Tannal benn metric bu baax ak benn anam bu baaxul balaa ngay saytu.
Doxal ab pilote bu ndaw ak ay done yu representatif, du ab demo bu leer.
Bindal fi hypothèse ticket loteri di jàppale ak fi pexe yu gëna yomba gëna baax.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Lottery Ticket Hypothesis quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Chinchilla - Tàggat yaram bu gëna baax
Laaj yi ñuy faral di laaj
Luy Hipothèse Ticket Loteri?
Ticket Lottery Hypothesis dafa wax ni ci biir benn reso neuronal bu mag, buñ tàmbali ci anam wu mujjee, dafa nëbb benn subnetwork bu ndaw - 'ticket buy jël ndam' - bi, tàggat boppam ci benn poids bi njëkk, mën méngoo ak njubteg reso bi yépp. Dafa am solo ndax dafay wane ni danuy tàggat parametre yu bari te ëpp li ñu soxla.
Luy 'tiketu ndam' ci xalaat bii?
Ticket biy jël ndam li mooy subnetwork bu ndaw bi, suñu ko tàggatee ci benn poids bi njëkk, dafay yegg ci njubte bu méngoo ak reso dense bi.
Lan moo tax tàggat subnetwork biñ dagg du mëna dem fileek defoo dara lu amul fenn?
Li gëna am solo mooy mask bu bariwul bi du dox ludul suñu ko boole ak initialisation original bi méngoo; reinitialisasioŋ aleatër daf koy dog.
Kan moo taxawal xalaatu biye lotari bi?
Jonathan Frankle ak Michael Carbin ñoo dugal xalaat bi ci seen këyitu MIT ci 2018.
Ban pexe lañuy jëfandikoo ngir am këyit yiy jël ndam li?
Dagg magnitude iteratif dafay tàggat lu bari, dindi poids yu magnitude bi gëna ndaw, ba noppi delloosi yeneen yi ci seen valeur yu njëkk.
Lan la hypothèse bi di wax ci reso yu yaatu yu bari ay parametre?
Gis-gis bi dafay tekki ni overparameterization dafay jàppale seetug subréseau bu néew buñ mëna tàggat, moo gën ñu soxla bépp poid.