InayofuataMwongozo unaofuata
Sheria za Kuongeza Mitandao ya Neural
Misingi
Lugha AI MWONGOZO
Sheria za Kupunguza Chinchilla
Chinchilla studied how to allocate a training-compute budget between parameters and training tokens.
Sikiliza mwongozo huuInaonyeshwa na AI · ~2 min
Katika ukurasa huudk 2 kusoma
Muhtasari
Its fitted results favor scaling both together in the evaluated regime; that objective is different from minimizing total deployment cost or guaranteeing one token ratio for every dataset.
Dive ya kina
The 2022 study fitted loss behavior across model and data budgets and estimated training-compute-optimal parameter and token allocations. The finding that both quantities should grow in similar proportions means comparable scaling exponents, not equal numerical counts. Chinchilla used about seventy billion parameters and 1.4 trillion training tokens, giving a twenty-token-per-parameter example. It was compared with the much larger Gopher under a comparable training-compute budget. The study argued that many evaluated contemporary large models were undertrained relative to the estimated optimum. These relationships are fitted results that depend on the data and setup; a fixed ratio is not a universal design law. LLaMA's paper discusses training smaller models longer as a way to consider inference efficiency, a different objective from training compute alone.
Utekelezaji wa Ulimwengu Halisi
Comparing the 70-billion-parameter Chinchilla experiment with the 280-billion-parameter Gopher baseline.
Using fitted training-budget relationships as a starting point rather than a universal ratio rule.
Distinguishing a from-scratch training optimum from a smaller model chosen for repeated inference.
Evaluating whether parameters or data provide more value under a defined compute and quality objective.
Mustakabali wa Sheria za Kuongeza Kiwango cha Chinchilla
Scaling research continues to examine data quality, repeated data, model architecture, and the cost of training versus serving. State the objective and evaluation conditions before calling an allocation optimal. A useful empirical trend is different from a guarantee that one parameter-token recipe dominates every workload.
Kuiweka katika vitendo
Kwa nini ni muhimu
- Kasi na kiwango. Mitiririko ya kazi ya lugha inaweza kusonga kwa kasi zaidi bila kuacha uthabiti.
- Kufikia na kufikia. Inapanua ufikiaji katika lugha na mitindo ya mawasiliano.
- Maamuzi ya wazi zaidi. Timu zinaweza kutumia muda mwingi kufanya uamuzi huku otomatiki ikishughulikia marudio.
Jihadharini na
- Mambo ya ukweli yanaweza kuingiza ripoti kwa utulivu, mitiririko ya usaidizi, au matokeo ya utafiti.
- Usikivu wa haraka unaweza kuunda matokeo yasiyolingana katika maombi sawa.
- Data nyeti ya maandishi inaweza kufichuliwa ikiwa vidhibiti vya ufikiaji ni dhaifu.
Hatua zinazofuata
- Bainisha umbizo la towe, toni na viwango vya ubora kabla ya kusambaza.
- Majibu ya msingi na vyanzo vinavyoaminika wakati wowote usahihi ni muhimu.
- Weka ukaguzi wa ukaguzi wa kibinadamu kwa matokeo ya juu.
- Fuatilia mifumo ya kushindwa na fundisha tena vidokezo au mtiririko wa kazi mara kwa mara.
Endelea Kuchunguza
Jarida halali
Pata taarifa ya kila siku ya AI
Hadithi tatu za AI zilizothibitishwa kila asubuhi ya wiki, zimetengenezwa kwa Kiingereza rahisi. Bila malipo daima, hakuna matangazo.
Barua pepe moja kila siku ya wiki. Jiondoe kwa bonyeza moja. Hatutauza au kushiriki anwani yako.
Jijaribu mwenyewe
Take the Chinchilla Scaling Laws quiz
Maoni ya haraka juu ya kila jibu, na cheti kinachoweza kushirikishwa na kitambulisho kinachoweza kuthibitishwa baada ya kupita kozi.
Changia elimu ya AI bila malipo. AI Understanding ni 501(c)(3) isiyo na faida — hakuna matangazo, hakuna kizuizi cha malipo, daima. Toa mchango
Maswali yanayoulizwa mara kwa mara
Sheria za Kupunguza Chinchilla ni nini?
Chinchilla studied how to allocate a training-compute budget between parameters and training tokens. Its fitted results favor scaling both together in the evaluated regime; that objective is different from minimizing total deployment cost or guaranteeing one token ratio for every dataset.
What does Chinchilla's finding that parameters and tokens should scale in similar proportions mean?
Similar proportional growth is not a literal one-token-per-parameter rule.
What token-per-parameter ratio does the 70B-parameter, 1.4T-token Chinchilla experiment illustrate?
1.4 trillion divided by seventy billion equals twenty; this example is not a universal optimum for every setup.
Je, Chinchilla alishinda mtindo gani licha ya kuwa mdogo zaidi?
Chinchilla ya kigezo cha 70B ilishinda Gopher ya DeepMind ya 280B kwa kutumia komputa sawa, kwa sababu ilipata mafunzo kwenye data nyingi zaidi.
Chinchilla ilimaanisha nini kuhusu mifano kama GPT-3 wakati huo?
Miundo mingi mikubwa ya enzi hiyo haikufunzwa, ikimaanisha kwamba wangefanya vyema zaidi na data zaidi kwa hesabu yao ya vigezo.
Takriban ni jinsi gani hesabu ilikadiriwa katika uchanganuzi wa Chinchilla?
Kokotoo la mafunzo (FLOPs) ni takriban sawia na idadi ya vigezo vinavyozidishwa na idadi ya tokeni za mafunzo.
Endelea kujifunza
Miongozo inayohusiana
Miongozo zaidi imechaguliwa kwa mada hii