Overview
Kudzika kaviri ndiko kucherekedza kunokatyamadza kuti sezvo modhi inokura, kukanganisa kwebvunzo kunotanga kuipa padhuze ne'interpolation threshold' asi zvobva zvaita zviri nani zvakare - zvichiramba iyo yekirasi yemabhuku tradeoff. Izvo zvine basa nekuti zvinobatsira kutsanangura kuti nei akakura, akawandisa neural network akazara zvakanaka pane kuwandisa.
Kaviri Descent Phenomenon inogara mukati meiyo AI toolkit. Paunonzwisisa, mamwe maAI misoro inova nyore kuongorora uye kuenzanisa.
Deep Dive
Classical statistics inodzidzisa U-shaped curve: sezvo modhi yakaoma inokwira, bvunzo kukanganisa kunodonha, bottoms kunze, yobva yasimuka sezvo modhi inodarika. Zvizvarwa zvakapetwa kaviri, zvakakurumbira naBelkin, Hsu, Ma, naMandal muna 2019 uye zvakadzidzwa pachiyero ne OpenAI, zvinoratidza kuti curve ine dzinza rechipiri. Chikanganiso chekuyedza chinokwira pachikumbaridzo chekududzira - poindi iyo modhi inongova nemaparamita akakwana kuti akwane pese pese pekudzidzira (zero dzidziso kukanganisa). Sunda kupfuura izvo muhutongi hwepamusoro uye chikanganiso chekuyedza chinowira zvakare, kazhinji pazasi peiyo classical inotapira nzvimbo. Mhedzisiro imwechete inoonekwa pakukura kwemuenzaniso, nguva yekudzidziswa ('epoch-wise' kudzika kaviri), uye saizi yedhata. Inodzokorora kutya kwekare kuti 'mamwe maparameter anogara anoreva kuwandisa.'
Technical Insight
Pachikumbaridzo chekududzira pane mhinduro imwechete inonyatsoenderana nedata, uye inomanikidzwa kuve yakajairwa uye yakakwirira-yakajairika, saka inowanda zvisina kunaka. Muhutongi hwepamusoro-soro, mhinduro zhinji dze zero-mhosho dziripo, uye rusaruro rwekudzika kwe gradient runotungamira kune yakatsvedzerera, yakaderera-yakajairika. Iyo yakasarudzika yepasi-yakaomesesa interpolators - kwete iyo parameter kuverenga pachayo - ndiyo inotyaira iyo yechipiri kudzika kudzikisa bvunzo kukanganisa.
Mastering Double Descent Phenomenon
Kudzika kaviri ndiko kucherekedza kunokatyamadza kuti sezvo modhi inokura, kukanganisa kwebvunzo kunotanga kuipa padhuze ne'interpolation threshold' asi zvobva zvaita zviri nani zvakare - zvichiramba iyo yekirasi yemabhuku tradeoff. Izvo zvine basa nekuti zvinobatsira kutsanangura kuti sei akakura, akawandisa neural network akazara zvakanaka pane kuwandisa. Kaviri Descent Phenomenon inogara mukati meiyo AI toolkit. Paunonzwisisa, mamwe maAI misoro inova nyore kuongorora uye kuenzanisa. Kuti uvake kunzwisisa kwakadzama, bata Double Descent Phenomenon semuenzaniso wekushandisa, kwete chinhu chimwe chete: tsanangura zvinodiwa, kujekesa fungidziro, uye patsanura zvinogona kuitwa nehurongwa hwakavimbika kubva kune zvichiri kuda kutonga kwenyanzvi.
Mukuita, zvikwata zvakasimba zvinoshandisa Double Descent Phenomenon zvinovaka mamodheru akasimba ekutanga, obva anyora iwo modhi kune zvimhingamipinyi zvekugadzira. Ivo vanonyora zvakajeka maitiro ebudiriro, bvunzo vachipokana ne data rechokwadi uye mafambiro ebasa, uye iterate zvichibva pane zvakacherechedzwa maitiro ekutadza kwete kuhwina-nguva imwe chete yebhenji. Apa ndipo apo kunzwisisa kwe theoretical kunoshanduka kuve kugona kwakasimba pane chigadzirwa, mutemo, uye mashandiro.
Inokubatsira kuparadzanisa zvakajeka zvichemo zvehunyanzvi kubva mumutauro wekushambadzira. Panguva imwecheteyo, Zvikwata zvakasiyana zvinogona kushandisa izwi rimwechete zvakasiyana, saka tsanangura nzvimbo nekukasira. Nzira yakatsiga ndeyekubatanidza kukurumidza kuyedza nekutonga: mhanyisa vatyairi vendege, tora humbowo, buritsa matanda esarudzo, uye urambe uchivandudza chengetedzo semaitiro emuenzaniso, zvinotarisirwa nemushandisi, uye zvinodikanwa zvekutonga.
Strategic Impact
Inokubatsira kuparadzanisa zvakajeka zvichemo zvehunyanzvi kubva mumutauro wekushambadzira.
Inokubatsira kuparadzanisa zvakajeka zvichemo zvehunyanzvi kubva mumutauro wekushambadzira. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Iwe unogona kubvunza zvirinani kuita mibvunzo usati washandisa mari kana nguva.
Iwe unogona kubvunza zvirinani kuita mibvunzo usati washandisa mari kana nguva. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Zvikwata zvine nzwisiso yakagovaniswa inoita zvirinani chigadzirwa, mutemo, uye sarudzo dzekudzidza.
Zvikwata zvine nzwisiso yakagovaniswa inoita zvirinani chigadzirwa, mutemo, uye sarudzo dzekudzidza. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Real-World Implementation
Kutsanangura kuti sei 175-bhiriyoni-parameta mutauro modhi ichiita zvirinani pane yakanyatso kurongedzerwa yepakati-saizi kunyangwe yakawanda yakawanda.
Kusarudza kudzidzisa kupfuura nzvimbo iyo kurasikirwa kwekusimbisa kunowedzera kwenguva pfupi, nekuti epoch-huchenjeri kudzika kaviri kunofanotaura kupora gare gare.
Kuongorora modhi yechiratidzo iyo kurongeka kwayo kwakanyura chaizvo kana parameter kuverenga ichienderana nekudzidziswa-seti saizi, wozoitungamira zvakadzika mukupfuura parameterization.
Kuzivisa modhi-saizi sarudzo mu AutoML kuitira kuti varapi vadzivise iyo isina kusimba yekududzira-chikumbaridzo zone.
Maitiro Ekuita
Kaviri Descent Phenomenon mukuita
Kutsanangura kuti sei 175-bhiriyoni-parameta mutauro modhi ichiita zvirinani pane yakanyatso kurongedzwa yepakati-saizi zvisinei nekuwanda kwakawanda.
Kutsanangura kuti sei 175-bhiriyoni-parameta mutauro modhi ichiita zvirinani pane yakanyatso kurongedzwa yepakati-saizi kunyangwe yakakura hukuru Matimu anowanzo kuwana mhedzisiro iri nani kana achinge atsanangura emhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi emupendero, uye kuteedzera zvese zvakawanikwa zvechigadzirwa uye kukanganisa mutengo nekufamba kwenguva.
Kaviri Descent Phenomenon mukuita
Kusarudza kudzidzisa kupfuura apo kurasikirwa kwekusimbisa kunowedzera kwenguva pfupi, nekuti epoch-huchenjeri kudzika kaviri kunofanotaura kupora gare gare.
Kusarudza kudzidzisa kupfuura iyo nzvimbo yekurasikirwa kwekusimbisa kunowedzera kwenguva pfupi, nekuti epoch-yakachenjera kudzika kaviri inofanotaura gare gare kudzoreredza Matimu anowanzo kuwana mhedzisiro iri nani kana achinge atsanangura emhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi ekumucheto, uye kuteedzera zvese zvakawanikwa zvechigadzirwa uye mutengo wekukanganisa nekufamba kwenguva.
Kaviri Descent Phenomenon mukuita
Kuongorora modhi yechiratidzo ine huchokwadi hwakanyura chaizvo kana parameter kuverenga ichienderana nekudzidziswa-seti saizi, wobva waitungamira zvakadzika mukupfuura parameterization.
Kuongorora modhi yechiratidzo iyo kurongeka kwayo kwakanyura chaizvo kana parameter kuverenga inoenderana nekudzidziswa-seti saizi, wobva waitungamira mukati mepamusoro-soro Matimu anowanzo kuwana mhedzisiro iri nani kana achinge atsanangura emhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi ekumucheto, uye kuteedzera zvese zvakawanikwa zvechigadzirwa uye kukanganisa mutengo nekufamba kwenguva.
Kaviri Descent Phenomenon mukuita
Kuzivisa modhi-saizi sarudzo muAutoML kuitira kuti varapi vadzivise iyo isina kusimba yekududzira-chikumbaridzo zone.
Kuzivisa modhi-saizi sarudzo muAutoML kuitira kuti varapi vadzivise iyo isina kusimba yekududzira-chikumbaridzo zone Matimu anowanzo kuwana mhedzisiro iri nani kana achinge atsanangura emhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi emupendero, uye kuteedzera zvese zvakawanikwa zvechigadzirwa uye mutengo wekukanganisa nekufamba kwenguva.
Njodzi & Guardrails
Zvikwata zvakasiyana zvinogona kushandisa izwi rimwechete zvakasiyana, saka tsanangura nzvimbo nekukurumidza.
Benchmarks inogona kutaridzika yakasimba nepo chaiyo-yenyika kuita isina kuenzana.
Kuregeredza mhando yedata uye zvirongwa zvekuongorora zvinowanzogadzira mhedzisiro isina kusimba.
Implementation Roadmap
Tanga netsanangudzo yemutauro wakajeka yemhedzisiro yaunoda.
Tanga netsanangudzo yemutauro wakajeka yemhedzisiro yaunoda. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Sarudza metric imwe yekubudirira uye imwe yekutadza mamiriro usati waedzwa.
Sarudza metric imwe yekubudirira uye imwe yekutadza mamiriro usati waedzwa. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Mhanya mutyairi mudiki ane data remumiriri, kwete demo rakakwenenzverwa.
Mhanya mutyairi mudiki ane data remumiriri, kwete demo rakakwenenzverwa. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Nyora apo Double Descent Phenomenon inobatsira uye uko nzira dzakareruka dziri nani.
Nyora apo Double Descent Phenomenon inobatsira uye uko nzira dzakareruka dziri nani. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.