Amategeko yo gupima Chinchilla
Amategeko yo gupima Chinchilla, kuva muri DeepMind mu 2022, yerekanaga ko indimi nini nini zavuzwe nabi: ku ngengo yimibare ihamye, ugomba gupima ingano yicyitegererezo hamwe namakuru yamahugurwa hafi yingana.
Incamake
It matters because it redefined what 'optimal' model size means and reshaped how labs spend compute.
Kwibira cyane
Mbere ya Chinchilla, icyerekezo cyari ukubaka moderi nini nini (nka 175B-parameter ya GPT-3) mugihe imyitozo kumubare muto ugereranije. DeepMind yahuguye moderi zirenga 400 mubunini na bije yamakuru, hanyuma ihuza umurongo uteganya igihombo nkigikorwa cyibipimo nibimenyetso munsi yingengo yimibare ihamye (FLOP). Ibisubizo byabo: ibipimo nibimenyetso byamahugurwa bigomba gupimirwa hamwe, hafi igipimo cya 1 kuri 1, bivuze ibimenyetso 20 byamahugurwa kuri buri kintu. Kugirango babigaragaze, bahuguye Chinchilla, moderi ya 70B yerekana ibipimo bya miriyoni 1,4, byarushije cyane 280B ibipimo bya Gopher nubwo yakoresheje mudasobwa imwe, kuko yatojwe amakuru menshi cyane.
Ubushishozi
Amategeko aturuka muguhuza imikorere yibihombo L (N, D) aho N ari ibipimo na D ni ibimenyetso, harimo-igihombo kidasubirwaho, urugero-ingano, hamwe namakuru-ingano yamakuru. Kugabanya igihombo ukurikije imbogamizi ya compte (compte igereranijwe hafi N inshuro D) itanga ibisubizo ko N na D nziza byombi bikura nkimbaraga zo kubara hamwe nibintu bisa, bityo rero compte-optimal iguma hafi yikimenyetso 20 kuri buri kintu.
Ingaruka z'Ingamba
Umuvuduko n'igipimo
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika.
Kugera no kugera
Yagura uburyo bwindimi nuburyo bwo gutumanaho.
Ibyemezo bisobanutse
Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo.
Ejo hazaza h'amategeko yo gupima Chinchilla
Chinchilla yahinduye umurima uva mukubara ibipimo kubara kugaburira moderi kure cyane yamakuru yo murwego rwohejuru, kandi moderi zigezweho akenshi zitoza neza kurenza 'compute-optimal' ingingo kugirango umwanzuro uhendutse. Nkuko urwego rwohejuru rwurubuga rwabaye ingume, kwitondera ni uguhindura amakuru, amakuru yubukorikori, ibihe byinshi, hamwe namakuru menshi kugirango akomeze gupima. Isomo ryibanze ryihanganira: amakuru nibipimo bigomba kuringanizwa, kandi ingano mbisi yonyine ntabwo ikiri intego.
Gushyira mu bikorwa Isi
Chinchilla ya 70M ya DeepMind itsinze 280B Gopher kubipimo ukoresheje compte ingana, mumahugurwa kumakuru menshi cyane
Kuyobora amakipe guteganya hafi 20 ibimenyetso byamahugurwa kuri buri kintu mugihe utegura icyitegererezo
Gutsindishiriza ibintu bito, bikungahaye kuri moderi nka LLaMA bihendutse gukora mugihe cyo gufata umwanzuro
Kugereranya niba icyitegererezo giteganijwe 'cyashizweho' kandi cyakungukira byinshi mumibare yinyongera kuruta ibipimo byiyongereye
Ingaruka & Kurinda
Ibintu bifatika bishobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi.
Kwihuta byihuse birashobora gukora ibisubizo bidahuye mubisabwa bisa.
Ibyanditswe byumvikana birashobora kugaragara niba kugenzura kugenzura ari ntege.
Igishushanyo mbonera
Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira.
Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro.
Komeza kugenzura abantu kugenzura ibisubizo byinshi.
Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Chinchilla Scaling Laws quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Amategeko yo gupima imiyoboro ya Neural
Ibibazo bikunze kubazwa
What is Chinchilla Scaling Laws?
Amategeko yo gupima Chinchilla, kuva muri DeepMind mu 2022, yerekanaga ko indimi nini nini zavuzwe nabi: ku ngengo yimibare ihamye, ugomba gupima ingano yicyitegererezo hamwe namakuru yamahugurwa hafi yingana. Nibyingenzi kuko yasobanuye icyo ingano yicyitegererezo 'optimal' isobanura kandi ihindura uburyo laboratoire ikoresha compte.
Ni ubuhe buryo bukuru bwagaragaye mu mategeko yo gupima Chinchilla?
Chinchilla yerekanye ko kuri bije ihamye yo kubara, ibipimo nibimenyetso byamahugurwa bigomba gukura hamwe, hafi 1-kuri-1.
Hafi ya bangahe ibimenyetso byamahugurwa kuri parameter Chinchilla yatanze byerekana ko ari byiza?
Ikigereranyo-cyiza gikora hafi ya 20 ibimenyetso byamahugurwa kuri buri kintu cyerekana icyitegererezo.
Ni ubuhe bwoko Chinchilla yarushije nubwo ari buto cyane?
70B-parameter Chinchilla yatsinze DeepMind yonyine 280B-parameter ya Gopher ikoresheje compte imwe, kuko yatoje amakuru menshi cyane.
Chinchilla yashakaga kuvuga iki kubijyanye na moderi nka GPT-3 icyo gihe?
Moderi nini nini yicyo gihe yarahawe imyitozo, bivuze ko bari gukora neza hamwe namakuru menshi yo kubara ibipimo byabo.
Hafi ya compte yagereranijwe gute mu isesengura rya Chinchilla?
Amahugurwa yo kubara (FLOPs) aragereranijwe numubare wibipimo byikubye numubare wamahugurwa.