Amategeko yo gupima imiyoboro ya Neural
Amategeko yo gupima ni formulaire ifatika yerekana ko igihombo cyurusobe rwimitsi igabanuka nkuko ugenda ukura ingano yicyitegererezo, ingano ya dataset, na compte.
Incamake
They matter because they let researchers forecast performance before spending millions on training a giant model.
Kwibira cyane
Amategeko yo gupima, yakunzwe na __AIU_PROTECTED_10 __ impapuro za 2020 na Kaplan na bagenzi be, yasanze gutakaza ibizamini bigabanuka nkamategeko agenga ingufu muburyo butatu: kubara ibipimo (N), ibimenyetso byamahugurwa (D), hamwe na compte yose (C). Byateguwe kuri log-log axe, igihombo na buri kintu kigize umurongo ugororotse uzenguruka ibyateganijwe byinshi. Umubano ufata uburyo bwo Gutakaza ≈ a + b · X ^ (- c), aho X ari ikintu gipima. Icy'ingenzi, umurimo wambere watanze ubunini bwikitegererezo bufite agaciro kuruta amakuru, bigatuma habaho irushanwa ryerekanwa nini cyane nka GPT-3 ya miliyari 175. Amategeko yo gupima yahinduye imyigire yimbitse kuva gukeka ihinduka disipuline yubuhanga iteganijwe, kureka amakipe akavuga ibisubizo binini biva mubushakashatsi buto, buhendutse.
Ubushishozi
Ifishi yimbaraga-amategeko isobanura buri kwiyongera kugwije kugwiza kubara bitanga umusaruro uhoraho wongeyeho igihombo. Igihombo gipimirwa mu njangwe cyangwa bits kuri buri kimenyetso cya cross-entropy. Kuberako ibyerekanwa c ari bito (akenshi hafi 0.05-0.1), inyungu nukuri ariko ziragabanuka: kubara kabiri bifasha cyane ugereranije no gukuba kabiri. Icy'ingenzi, aya mategeko asobanura igihombo kidasubirwaho-cyongeweho-kigabanywa, aho ijambo rihoraho rifata entropie yimbere yamakuru nta moderi ishobora gutsinda.
Ingaruka z'Ingamba
Ibyemezo bisobanutse
Iragufasha gutandukanya ibyifuzo bya tekiniki bisobanutse nururimi rwo kwamamaza.
Igiciro na bije
Urashobora kubaza ibibazo byiza byo gushyira mubikorwa mbere yo gukoresha amafaranga cyangwa igihe.
Itsinda hamwe nakazi
Amakipe asangiye ibitekerezo akora ibicuruzwa byiza, politiki, nibyemezo byo kwiga.
Ejo hazaza h'amategeko yo gupima imiyoboro ya Neural
Abashakashatsi barimo kwagura amategeko yo gupima ibirenze gutakaza igihombo kumurimo wo hasi wukuri, moderi nyinshi, hamwe nigihe cyo kubara, aho moderi yo gutekereza ikoresha ibitekerezo byinshi kuri buri kibazo. Nkuko inyandiko-yujuje ubuziranenge iba ingume, kwitondera guhindukirira ubwiza bwamakuru, amakuru yubukorikori, hamwe namategeko yo gupima amakuru. Bamwe bavuga ko igipimo fatizo gikubita imipaka ifatika, imbaraga, hamwe ninyandiko iboneka, bigatuma umurima ugana imikorere ya algorithmic nuburyo bushya aho kubaka binini.
Gushyira mu bikorwa Isi
Guteganya igihombo cyanyuma cyateganijwe miliyari 70-ziteganijwe kuva murukurikirane rwibizamini bito-miliyoni 100-ibipimo mbere yo gukora ingengo yimari ya GPU.
Guhitamo umubare wa trillioni yikimenyetso cyo gukusanya kugirango ingengo yimibare ihamye idaseswa kurugero rwabigenewe.
Kugereranya ibyubatswe bibiri bihendutse muguhuza umurongo wapimye ku gipimo gito aho guhugura byombi mubunini bwuzuye.
Gushiraho ibiteganijwe neza kubashoramari cyangwa gutanga abasesengura mugukwirakwiza igihombo kumurongo kurwego rwo kubara.
Ingaruka & Kurinda
Amakipe atandukanye arashobora gukoresha ijambo rimwe muburyo butandukanye, sobanura intera hakiri kare.
Ibipimo birashobora kugaragara bikomeye mugihe imikorere-yisi-itaringaniye.
Kwirengagiza ubuziranenge bwamakuru na gahunda yo gusuzuma akenshi bitanga ibisubizo byoroshye.
Igishushanyo mbonera
Tangira nururimi rusobanutse rwibisubizo ukeneye.
Toranya intsinzi imwe hamwe nuburyo bumwe bwo gutsindwa mbere yo kwipimisha.
Koresha umuderevu muto hamwe namakuru ahagarariye, ntabwo ari demo yashizweho.
Inyandiko aho Amategeko yo gupima imiyoboro ya Neural ifasha kandi nuburyo bworoshye bworoshye.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Scaling Laws for Neural Networks quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Imiyoboro ya Neural Network
Ibibazo bikunze kubazwa
What is Scaling Laws for Neural Networks?
Amategeko yo gupima ni formulaire ifatika yerekana ko igihombo cyurusobe rwimitsi igabanuka nkuko ugenda ukura ingano yicyitegererezo, ingano ya dataset, na compte. Ntacyo bitwaye kuko bareka abashakashatsi bakavuga imikorere mbere yo gukoresha miriyoni mugutoza icyitegererezo kinini.
Kuri log-log axes, nigute gutakaza ikizamini mubisanzwe bitwara nkuko compte yiyongera mumategeko yo gupima?
Gutakaza na compte, ingano yicyitegererezo, cyangwa data ikora umurongo wegeranye ugororotse kumurongo wibiti, umukono wububasha-amategeko.
Ni ubuhe buryo butatu amategeko yambere yo gupima afitanye isano nigihombo?
Kaplan n'abandi. yize igihombo nkamategeko yimbaraga mubare (N), ibimenyetso byamahugurwa (D), hamwe na compte yose (C).
Kuki amategeko yo gupima afite agaciro cyane muri laboratoire ya AI?
Muguhuza umurongo ku gipimo gito, amakipe ateganya imikorere yikitegererezo mbere yo gukoresha amafaranga menshi.
Ijambo rihoraho (ridashoboka) muburyo bwo gutakaza amategeko agereranya iki?
Igihombo gifite igice kigabanuka kigabanuka nubunini hiyongereyeho igorofa idasubirwaho yashyizweho namakuru yihariye.
Kuki inyungu zingana zisobanurwa nk 'kugabanuka'?
Kuberako imbaraga-amategeko yerekana ari nto, kugereranya kugwiza bingana byongera umusaruro gahoro gahoro kugabanuka rwose.