AMABWIRIZA Yibanze

Grokking no Gutinda Rusange

Grokking asobanura imanza zize aho zifatirwa muri rusange zitera imbere nyuma y'urusobe ruhuye nurugero rwamahugurwa.

Umva iki gitaboAI-byavuzwe · ~3 min

Kuriyi page3 min soma
  1. Incamake
  2. Kwibira cyane
  3. Gushyira mu bikorwa Isi
  4. Ejo hazaza ha Grokking no Gutinda Rusange
  5. Kubishyira mubikorwa
  6. Komeza Ubushakashatsi
  7. Ibibazo bikunze kubazwa

Incamake

Imirimo mito ya algorithmic itanga igeragezwa risobanutse, ariko ntabwo ari plateau yemewe cyangwa imyitozo ndende iremeza ko iyi nzibacyuho izaba.

Kwibira cyane

Impapuro zumwimerere zoherejwe, zatanzwe kumugaragaro muri Mutarama 2022, ziga imibare mito ya algorithm yakozwe. Mubigeragezo bimwe, umuyoboro uhuza amakuru yamahugurwa mugihe ibikorwa-bigumaho bikomeza kuba bibi, hanyuma bigahinduka nyuma yuburyo bwiza cyane. Ibi bitandukanya ingero ziboneye nimyitwarire yo kwiga yimura ingero zitagaragara zumurimo wibanze. Ntabwo bivuze ko buri cyitegererezo gifite hafi-zeru kwemeza neza mugihe cya kibaya cyambere cyangwa ko gufata mumutwe bigomba guhora bicika burundu. Ubushakashatsi bwumwimerere buratandukanya ingano ya dataset no guhitamo neza, harimo kubora. Ibyavuye mu bushakashatsi bireba ibyo byashizweho aho kuba ibisobanuro byihariye kuri rusange. Iterambere ryibipimo bya Grokking binyuze muri Mechanistic Interpretability ikora iperereza kuri transformateur nto zahuguwe kubwinyongera. Algorithm yize ikoresha imiterere ya Fourier hamwe na trigonometric indangamuntu kugirango yerekane kwiyongera binyuze mumibanire izenguruka. Abashakashatsi basuzuma ibikorwa nuburemere kandi bagakoresha Fourier-umwanya wo gupima konti yabo. Basobanura gufata mu mutwe, gushinga imizunguruko, no gusukura ibyiciro, hamwe nuburyo rusange bugaragara mbere yo gusimbuka kugaragara mubizamini byukuri. Kubora ibiro bikuraho gufata mu mutwe mugihe cyo gukora isuku. Ibi birasobanutse neza kuruta inkuru aho algorithm yose igaragara gitunguranye mugihe cyo gusimbuka neza. Imirimo isanzwe itanga ibirango nyabyo: kurugero, kongeramo itandatu na gatanu modulo irindwi itanga bine kuko amababi cumi numwe asigaye ane nyuma yo kugabana na barindwi. Urupapuro rwumwimerere ruvuga kandi ko byongerewe ibisabwa kugirango amahugurwa yamakuru abe mato. Ubushakashatsi nk'ubwo bugenzurwa butera ubundi bushakashatsi, ariko ntugaragaze ko moderi nini yoherejwe amaherezo izatera imbere niba yatojwe ubuziraherezo.

Gushyira mu bikorwa Isi

Gutegura imyitozo no gufata neza ukuri kubigeragezo bya arithmetic.

Kugereranya igenamiterere risanzwe kuri dataset ntoya isobanuwe.

Kugenzura umuzenguruko ushingiye kuri Fourier wize muri transformateur nto.

Guhindura imyitozo-yashizeho ingano mugihe wandika optimizasiyo ikenewe kubikorwa byakozwe.

Ejo hazaza ha Grokking no Gutinda Rusange

Ubushakashatsi bugenzurwa bushobora kugerageza uburyo ingano ya dataset, ubwubatsi, guhora, hamwe no gutezimbere bigira ingaruka kubitinda rusange. Ingamba ziterambere zishingiye kumuzunguruko wagenzuwe zishobora kwerekana imyitwarire ihishe kumurongo wuzuye, nubwo gupima isesengura kuri moderi nini bikomeje kuba ikibazo gifunguye. Abashakashatsi bagomba gutandukanya uburyo bwapimwe mubikorwa bito nibisobanuro byatanzwe kuri buri cyitegererezo. Guhagarika ibyemezo bifatika biracyakenewe gusuzumwa, imbogamizi zumutungo, nibimenyetso byiterambere; kubaho kwa grokking bitanga impamvu yo gukora iperereza kuri plateaus aho kuba itegeko ryo gukomeza imyitozo yose ubuziraherezo.

Kubishyira mubikorwa

Impamvu ari ngombwa

  • Ibyemezo bisobanutse. Iragufasha gutandukanya ibyifuzo bya tekiniki bisobanutse nururimi rwo kwamamaza.
  • Igiciro na bije. Urashobora kubaza ibibazo byiza byo gushyira mubikorwa mbere yo gukoresha amafaranga cyangwa igihe.
  • Itsinda hamwe nakazi. Amakipe asangiye ibitekerezo akora ibicuruzwa byiza, politiki, nibyemezo byo kwiga.

Witondere

  • Amakipe atandukanye arashobora gukoresha ijambo rimwe muburyo butandukanye, sobanura intera hakiri kare.
  • Ibipimo birashobora kugaragara bikomeye mugihe imikorere-yisi-itaringaniye.
  • Kwirengagiza ubuziranenge bwamakuru na gahunda yo gusuzuma akenshi bitanga ibisubizo byoroshye.

Intambwe ikurikira

  1. Tangira nururimi rusobanutse rwibisubizo ukeneye.
  2. Toranya intsinzi imwe hamwe nuburyo bumwe bwo gutsindwa mbere yo kwipimisha.
  3. Koresha umuderevu muto hamwe namakuru ahagarariye, ntabwo ari demo yashizweho.
  4. Inyandiko aho Grokking na Gutinda Rusange bifasha kandi nuburyo bworoshye bworoshye.

Komeza Ubushakashatsi

Ibaruwa y'amakuru y'ubuntu

Ibuka amakuru ya buri munsi ya AI

Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.

Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.

Isuzume

Fata ikibazo cya Grokking kandi cyatinze

Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.

Tangira ikibazo

Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano

Ibibazo bikunze kubazwa

Grokking nubukererwe rusange ni iki?

Grokking asobanura imanza zize aho zifatirwa muri rusange zitera imbere nyuma y'urusobe ruhuye nurugero rwamahugurwa. Imirimo mito ya algorithmic itanga igeragezwa risobanutse, ariko ntabwo ari plateau yemewe cyangwa imyitozo ndende iremeza ko iyi nzibacyuho izaba.

Ni ubuhe buryo bwo gukora buranga ubushakashatsi bwakozwe?

Gutinda bireba rusange kurenza ingero zagaragaye munsi yize.

Nibihe bikorwa urupapuro rwumwimerere rwakoresheje nkurwego rugenzurwa?

Impapuro rusange zatanzwe muri Mutarama 2022 kandi ziga imibare mito ya algorithmic.

Ni ubuhe buryo bwo kwishora mubikorwa byari byasuzumwe mubushakashatsi bwambere bwo gufata?

Ubushakashatsi bwumwimerere bukora iperereza kubora no guhitamo neza; nta mpamvu n'imwe ihuriweho na bose.

Ni ubuhe buryo bwo kubara bwatanzwe bwa modular-yongeyeho ubukanishi bwerekanye muri transformateur ntoya yatojwe?

Isesengura na Fourier gukuraho bishyigikira algorithm yihariye yize.

Ni ikihe cyiciro cyo gutondekanya ingamba ziterambere kuri Grokking zisobanura mubikorwa byamahugurwa yize?

Imiterere yumuzunguruko mbere yisuku nyuma ijyanye nikizamini kigaragara-cyukuri.