Grokking ak yeexal
Grokking lu yéeme la, fu reso neuronal bi njëkka xam done yi mu tàggat, mu toog ci njubte bu jege-zero ci diir bu yàgg, ba noppi ci saasi mu generalise lu yàgg ginaaw bi njubteg tàggat bi yeggee 100%.
Résumé
It overturns the intuition that learning and generalization happen together.
Plongeur bu xóot
__AIU_PROTECTED_10_ Gëstukat yi ñoo ko gis ci 2021 ci ay liggéey yu ndaw yu melni arithmétique modulaire, grokking dafay wane courbe bu am ñaari fase yu ñaw. Ci ndoorte li, xeetu model bi méngoo na bu baax ak tàggat yaram bi, ci noonu la performance validation bi di dess ci chance, di nuru lu amul benn yaakaar. Ginaaw loolu, ginaaw ay junni wala sax ay milioŋ ci yeneen jéego yu amul benn jéego bu leer, njubte gi ci saasi dafay tëb ba jege-baax. Faram-fàcce bi gëna am solo mooy ni diisaay bi di wàññeeku (regularization) dafay sonal ndànk reso bi ngir mu bàyyi ab pexe buñu xam bu yomba dagg ba noppi gis benn bu dëgër, buñ yamale buy jàpp sàrt yi ci suuf, ci misaal di wane yokk modular ni ay wëréelu ci kaw benn cercle. Grokking dafay gëna feeñ ci ensemble done yu ndaw, waaye xam ko dafay leer ci mekanik yu gëna xóot ci kañ ak lu tax generalisation di feeñ.
Gis-gis xarala
Jàngat mekanik yi dañu soppi reso grokked yi, ñu gis ni dañuy jëfandikoo algorithm yu sell, lu ci melni jëfandikoo ay embedding yu nuru Fourier ngir def arithmétique modulaire jaaraleko ci dàntite trigonometrik. Jaar-jaar bi mingi méngoo ak diisaayu reso bi di gëna néew, di gëna wàññeeku ci wàllu regularisasioŋ: memorisation dafa soxla diisaay yu yaatu te jaxasoo, waaye circuit generalisation bi moo gëna yomba def. Kon Grokking dafay wane joŋante bi am ci digganté ab pexe ci xel bu gaaw ci gis ak ab pexe bu yeex defar, bu gëna am njariñ ci yamale.
njeextalu pexe
dogal yu gëna leer
Daf lay jàppale nga tàqale kàddu yu leer ci wàllu xarala ak làkku fësal njaay.
Njëgg ak budget
Mën nga laaj laaj yu gëna baax ci samp gi balaa ngay dugal xaalis wala sa jotu liggéey.
Ekip ak def liggéey
Ekip yi bokk xam-xam ñoo gëna mëna jël yenn dogal ci wàllu produit, politik ak jàng.
Ëlëgu Grokking ak yeexal jëmmal
Grokking palanteer la buy dugg ci science generalisation bi gëstukat yi di am yaakaar yokk ko. Laaj yi ubbeeku ñooy ndax generalisation bu yeex dafay am ci noppi ci biir model yu mag, naka lañuy gisee wala gaawlu jaar-jaar bi, ak li muy tekki ngir xam su model bi jàngee dëgg-dëgg ab konsept ak misaal yuñ memorise. Insights mën na tax ñu gëna mëna yamale mbir yi, oraaru tàggat yaram, ak jumtukaayi tekki làkk, te mën na tax ñu mëna wax luy xew ci misaali làkk yu yaatu.
Doxal ci àdduna dëgg
Jàngat liggéeyu arithmétik modular ngir soppi ingenieur yi ci sircuit yi reso bi di jàng
Wane ni poid bi di wàññeeku dafay joge ci memorisasioŋ dem ci generalisasioŋ dëgg
Yegle gëstu tekki ci joxe xeetu doxalin bu sell, xam bu baax ngir jàngat
Àrtu praktiseur yi ni plateau validation teel du saa yu nekk tekki ni model bi daa ñàkk jàng
Risk yi ak balustrade yi
Ekip yu bari mën nañu jëfandikoo benn baat ci anam wu wuute, kon teela leeral yaatuwaayam.
Benchmark yi mën nañu nuru lu am doole waaye performance yi ci àdduna bi duñu tolloo.
Bëgg kalite done ak palaŋu jàngat dafay faral di jur njariñ yu yomba dagg.
Roadmap ngir samp gi
Tàmbaleel ci joxe leeral ci làkk wu leer ci njariñ li nga soxla.
Tannal benn metric bu baax ak benn anam bu baaxul balaa ngay saytu.
Doxal ab pilote bu ndaw ak ay done yu representatif, du ab demo bu leer.
Dokumenteer fi Grokking ak Generalisation bu yeex di jàppale ak fi pexe yu gëna yomba gëna baax.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Grokking and Delayed Generalization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Yamale
Laaj yi ñuy faral di laaj
What is Grokking and Delayed Generalization?
Grokking lu yéeme la, fu reso neuronal bi njëkka xam done yi mu tàggat, mu toog ci njubte bu jege-zero ci diir bu yàgg, ba noppi ci saasi mu generalise lu yàgg ginaaw bi njubteg tàggat bi yeggee 100%. Dafay soppi xalaat biy wax ni jàng ak yamale dañuy ànd.
Lan mooy grokking ci tàggat reso neuronal?
Grokking mooy tëb ci lu gaaw ba ci performance validation bu baax biy am ginaaw bi tàggat yaram jotee 100%.
Ci ban xeetu liggéey lañu njëkka gis grokking?
Gëstukat yi xamlenañu ni grokking ci 2021 ci jafe-jafe algorithm yu ndaw yu melni yokk modular.
Lan lañu gëna wax ni moo waral ñu jàll ci generalisation ci grokking?
Decay bu diisaay bi dafay puus reso bi ndànk-ndànk, joge ci ab pexe buñu xam bu yomba dagg, mu dem ci ab sircuit bu kompact, buy yamale.
Bi gëstukat yi defaree ingenieur bu bees ci reso bu grokked ci arithmétique modulaire, lan lañu ci gis?
Tekkib mekanik bi dafa wane ni reso bi jàng na trigonometrik, di wane ay wërngël ngir xayma arithmétique modulaire.
Lan moo waral ñuy wax grokking ni ñaari pexe yuy xeex?
Reseau yi dañuy gaaw am pexe ngir mëna jàng, waaye suñu nekkee ci regularisation, dañuy mujjee nekk ci circuit generalisation bu gëna yomba def.