Grokking uye Kunonoka Generalization
Grokking chinhu chinokatyamadza apo neural network inotanga kubata nemusoro data rayo rekudzidziswa, inogara padhuze-zero kurongeka kwechokwadi kwenguva yakareba, uye kamwe kamwe inogadzirisa nguva refu mushure mekudzidziswa kwechokwadi kwarova 100%.
Pfupiso
It overturns the intuition that learning and generalization happen together.
Kudzika Kwakadzika
Zvakawanikwa ne OpenAI vatsvakurudzi muna 2021 pamabasa madiki algorithmic se modular arithmetic, grokking inoratidza yakapinza-pase-pase curve. Pakutanga, modhi yacho inokodzera kudzidziswa kwakaiswa zvakakwana nepo kusimbiswa kwekuita kunoramba kuri pamukana, kutaridzika zvisina tariro. Zvino, mushure mezviuru kana mamirioni ematanho ekuwedzera pasina kufambira mberi kuri pachena, chokwadi chechokwadi chinosvetuka kusvika pedyo-chakakwana. Tsananguro inotungamira ndeyekuti kuora kwehuremu (kudzoreredza) zvishoma nezvishoma kunomanikidza network kuti isiye mhinduro yemusoro uye kuwana compact, yakarongeka iyo inonyatso tora iyo yepasi mutemo, semuenzaniso inomiririra modular yekuwedzera sekutenderera padenderedzwa. Grokking inonyanya kuoneka pamadiki ekugadzira datasets, asi kuinzwisisa kunovhenekera pakadzika mechanics enguva uye nei generalization inobuda.
Technical Insight
Mechanistic zvidzidzo zvinodzosera kumashure-engineered grokked network uye vakawana kuti vanoshandisa algorithms akachena, sekushandisa Fourier-senge denderedzwa embeddings kuita modular arithmetic kuburikidza netrigonometric identities. Shanduko inoenderana nehuremu hwenetiweki huremu uye huchidzikira-pasi pasi penguva dzose: kuyeuka kunoda hukuru, huremu husina kurongeka, nepo generalizing wedunhu iri nyore. Saka Grokking inoenzanisira makwikwi pakati pemhinduro yekukurumidza-kuwana-yemusoro uye inononoka-kune-fomu, inonyatso shanda kuumba imwe.
Strategic Impact
Sarudzo dzakajeka
Inokubatsira kuparadzanisa zvakajeka zvichemo zvehunyanzvi kubva mumutauro wekushambadzira.
Mutengo uye bhajeti
Iwe unogona kubvunza zvirinani kuita mibvunzo usati washandisa mari kana nguva.
Team uye workflow
Zvikwata zvine nzwisiso yakagovaniswa inoita zvirinani chigadzirwa, mutemo, uye sarudzo dzekudzidza.
Ramangwana reGrokking uye Kunonoka Kugadziriswa
Grokking ihwindo mune sainzi ye generalization iyo vaongorori vanotarisira kukwira. Mibvunzo yakavhurika inosanganisira kana kunonoka kuita kwese kunoitika chinyararire mukati memhando huru, maitiro ekuona kana kukurumidza kuchinja, uye zvazvinoreva pakuziva kana modhi yakanyatso dzidza pfungwa maringe nemienzaniso yakadzidzwa nemusoro. Insights inogona kuzivisa zvirinani kugadzwa, zvirongwa zvekudzidzisa, uye maturusi ekududzira, uye inogona kubatsira kufanotaura hunyanzvi hwekubuda mumhando dzemitauro mikuru.
Real-World Implementation
Kudzidza modular arithmetic mabasa ekudzosera kumashure-injiniya iwo chaiwo maseketi anodzidziswa netiweki
Kuratidza kuti kuora kwehuremu kunofambisa sei kuchinja kubva mumusoro kuenda kuhuwandu hwechokwadi
Kuzivisa kududzira tsvakiridzo nekupa yakachena, inonzwisiswa zvizere modhi maitiro ekuongorora
Kuyambira varapi kuti nzvimbo dzekutanga dzekusimbisa hazvireve kuti modhi yatadza kudzidza
Njodzi & Guardrails
Zvikwata zvakasiyana zvinogona kushandisa izwi rimwechete zvakasiyana, saka tsanangura nzvimbo nekukurumidza.
Benchmarks inogona kutaridzika yakasimba nepo chaiyo-yenyika kuita isina kuenzana.
Kuregeredza mhando yedata uye zvirongwa zvekuongorora zvinowanzogadzira mhedzisiro isina kusimba.
Implementation Roadmap
Tanga netsanangudzo yemutauro wakajeka yemhedzisiro yaunoda.
Sarudza metric imwe yekubudirira uye imwe yekutadza mamiriro usati waedzwa.
Mhanya mutyairi mudiki ane data remumiriri, kwete demo rakakwenenzverwa.
Gwaro uko Grokking uye Delayed Generalization inobatsira uye uko nzira dzakareruka dziri nani.
Ramba Uchiongorora
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Grokking and Delayed Generalization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gaidhi rinotevera
Regularization
Mibvunzo inowanzo bvunzwa
What is Grokking and Delayed Generalization?
Grokking chinhu chinokatyamadza apo neural network inotanga kubata nemusoro data rayo rekudzidziswa, inogara padhuze-zero kurongeka kwechokwadi kwenguva yakareba, uye kamwe kamwe inogadzirisa nguva refu mushure mekudzidziswa kwechokwadi kwarova 100%. Iyo inopidigura iyo intuition yekuti kudzidza uye generalization zvinoitika pamwechete.
Chii chinotsanangura grokking mune neural network kudzidziswa?
Grokking ndiko kusvetuka kamwe kamwe kune yakanaka yekusimbisa kuita kunoitika zvakanaka mushure mekudzidziswa kwechokwadi kwatova pa100%.
Pamhando ipi yemabasa kwakatanga kuonekwa zvakanyanya?
OpenAI vatsvakurudzi vakashuma grokking muna 2021 pamatambudziko madiki ekugadzira algorithmic senge modular kuwedzera.
Ndechipi chinhu chinonyanya kurumbidzwa nekufambisa shanduko kune generalization mu grokking?
Kuora kwehuremu zvishoma nezvishoma kunosundira network kure neyakavhurika nemusoro mhinduro yakananga kune compact, generalizing dunhu.
Vatsvagiri pavakadzosera-injiniya network yakashata pane modular arithmetic, chii chavakawana?
Kududzirwa kwemichina kwakaratidza network yakadzidza trigonometric, denderedzwa inomiririra kuverengera modular arithmetic.
Sei grokking ichitsanangurwa semakwikwi pakati pemhinduro mbiri?
Manetiweki anokurumidza kuwana mhinduro yemusoro asi, pasi penguva dzose, anozopedzisira asangana kune yakareruka generalizing wedunhu.