Chinchilla Kubara-Amahugurwa meza
Chinchilla ni 2022 DeepMind isanga urugero rwindimi nini rwarakozwe nabi: kubwingengo yimibare ihamye ugomba gupima ibipimo hamwe namakuru hafi kimwe, ntabwo wubaka icyitegererezo kinini.
Incamake
It reshaped how the industry balances model size against training data.
Kwibira cyane
Impapuro za Chinchilla za DeepMind zasubiwemo gupima kandi zitoza moderi zirenga 400 kugirango zibone compte-nziza. Umutwe wumutwe wintoki: ingano yicyitegererezo hamwe nibimenyetso byamahugurwa bigomba gukura mugufunga, ibimenyetso 20 byamahugurwa kuri buri kintu. Kugirango babigaragaze, bahuguye Chinchilla, moderi ya miliyari 70-yerekana ibipimo bya miriyoni 1,4, bakoresheje compte imwe na miliyari 280-ya Gopher yatojwe ku bimenyetso bike cyane. Chinchilla, nubwo ari ntoya inshuro enye, yarushije Gopher, GPT-3, n'ibindi bihangange hafi ya byose. Isomo ryatesheje agaciro umwanzuro OpenAI umwanzuro ushimangira ubunini burenze amakuru, werekana moderi nyinshi zamamaye zasize imikorere kumeza kuba nini cyane kandi ifite inzara.
Ubushishozi
Chinchilla igihombo gikwiye nka L (N, D) = E + A · N ^ (- α) + B · D ^ (- β), hamwe na α na β byombi hafi 0.34, bivuze ibipimo namakuru bitanga hafi muburyo bumwe. Kunonosora ibi munsi yimibare ihamye (kubara ≈ 6 · N · D kubahindura) bitanga ibisubizo bingana. Icyitegererezo gito, gikungahaye ku makuru nacyo gihendutse gukora ku mwanzuro, bityo inyungu zacyo zikomatanya mu kohereza, ntabwo ari imyitozo gusa.
Ingaruka z'Ingamba
Ibyemezo bisobanutse
Iragufasha gutandukanya ibyifuzo bya tekiniki bisobanutse nururimi rwo kwamamaza.
Igiciro na bije
Urashobora kubaza ibibazo byiza byo gushyira mubikorwa mbere yo gukoresha amafaranga cyangwa igihe.
Itsinda hamwe nakazi
Amakipe asangiye ibitekerezo akora ibicuruzwa byiza, politiki, nibyemezo byo kwiga.
Kazoza ka Chinchilla Kubara-Amahugurwa meza
Moderi zigezweho nka Llama 3 zisunika nkana kure ya Chinchilla 20-tokens-per-parameter, guhugura moderi ntoya kuri trillioni yikimenyetso kugirango ifate umwanzuro uhendutse, yemera compte yimyitozo ya suboptimal. Nkuko amakuru meza akura gake, inyungu ziragenda ziyongera mubihe byinshi, amakuru yubukorikori, hamwe no kuyungurura ubuziranenge. Chinchilla ikomeje kuba ingingo, ariko icyiza kiragenda giterwa nigiciro cyo kugereranya ubuzima, ntabwo ari ingengo yimari yigihe kimwe gusa.
Gushyira mu bikorwa Isi
Guhitamo guhugura moderi ya miliyari 7-yerekana kuri tiriyari 2 aho kuba miliyari 30 kuri moderi nkeya kuri data nkeya kuri bije imwe.
Ugereranije ko miliyari 10-yerekana imiterere yifuza ko miliyari 200 zigera kuri compte-nziza nziza.
Gutsindishiriza icyitegererezo gito cyoherejwe kugirango ugabanye buri kibazo cyibiciro mugihe uhuza ubuziranenge bwa mukeba.
Kugenzura icyitegererezo gihari no kurangiza cyarakozwe, hanyuma utegure imyitozo ndende ikora aho kongera ibipimo.
Ingaruka & Kurinda
Amakipe atandukanye arashobora gukoresha ijambo rimwe muburyo butandukanye, sobanura intera hakiri kare.
Ibipimo birashobora kugaragara bikomeye mugihe imikorere-yisi-itaringaniye.
Kwirengagiza ubuziranenge bwamakuru na gahunda yo gusuzuma akenshi bitanga ibisubizo byoroshye.
Igishushanyo mbonera
Tangira nururimi rusobanutse rwibisubizo ukeneye.
Toranya intsinzi imwe hamwe nuburyo bumwe bwo gutsindwa mbere yo kwipimisha.
Koresha umuderevu muto hamwe namakuru ahagarariye, ntabwo ari demo yashizweho.
Inyandiko aho Chinchilla Compute-Optimal Training ifasha kandi nuburyo bworoshye bworoshye.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Chinchilla Compute-Optimal Training quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Amahugurwa y'Igihe-Igihe
Ibibazo bikunze kubazwa
What is Chinchilla Compute-Optimal Training?
Chinchilla ni 2022 DeepMind isanga urugero rwindimi nini rwarakozwe nabi: kubwingengo yimibare ihamye ugomba gupima ibipimo hamwe namakuru hafi kimwe, ntabwo wubaka icyitegererezo kinini. Yahinduye uburyo inganda zingana ingano yicyitegererezo namakuru yamahugurwa.
Ni irihe tegeko rizwi cyane rya Chinchilla ryo gukurikiza amahugurwa meza?
DeepMind yasanze ibipimo nibimenyetso bigomba gupimirwa hamwe hafi 20 ibimenyetso kuri buri kintu kuri bije yatanzwe.
Nigute 70B-parameter Chinchilla yagereranije na 280B-ya Gopher?
Yatojwe ku bimenyetso byinshi hamwe na compte imwe, Chinchilla yatsinze Gopher nini cyane mubipimo byinshi.
Ni ikihe kibazo cy'ingenzi impapuro za Chinchilla zagaragaje kuri moderi nini zabanjirije iyi?
Moderi nka GPT-3 na Gopher yari nini cyane ugereranije namakuru yabo yo guhugura, hasigara imikorere idashoboka.
Hafi ya bangahe amategeko ya Chinchilla yerekana moderi ya miliyari 10-ibipimo?
Ku bimenyetso 20 kuri buri kintu, miliyari 10 ibipimo byerekana ibimenyetso bigera kuri miliyari 200.
Usibye gukora neza imyitozo, ni ukubera iki icyitegererezo gito, gikungahaye ku makuru gikurura gahunda?
Ibipimo bike bisobanura hasi kuri buri kibazo, bityo igiteranyo cyo kuzigama igihe cyose icyitegererezo cyakoreshejwe.