quiz bu lëkkale ak tegtal · Diggante Niveau

Chinchilla

Saytu Chinchilla tàggat-budget gis-gis, 70B / 1.4T misaal, escale proportion, jege xayma, ak wuute gi ak jëfandikoo gi gëna xéewale.

Yooni njiit yu jëm ciYoon yiy yamale Chinchilla
Laaj 1 de 8

What does Chinchilla's finding that parameters and tokens should scale in similar proportions mean?

Xoolaatal laaj bu nekk8 laaj

Sooy def quiz bi, tontu yi ak leeral yi dañuy feeñ.

  1. What does Chinchilla's finding that parameters and tokens should scale in similar proportions mean?
    • Both should grow with comparable scaling exponents under the studied training-compute objective
    • The numerical token count must equal the parameter count
    • Training data does not affect the fitted loss
    • Every deployment must choose the largest possible model
  2. What token-per-parameter ratio does the 70B-parameter, 1.4T-token Chinchilla experiment illustrate?
    • Exactly one token per parameter
    • About two thousand tokens per parameter
    • About twenty training tokens per parameter
    • A ratio determined only by the number of GPUs
  3. Ban xeetu Chinchilla moo gëna am doole doonte moo gëna ndaw?
    • GPT-2
    • Gofer
    • Bert
    • Mamba
  4. Lan la Chinchilla bëgga wax ci model yu melni GPT-3 ci jamono jooju?
    • Dañu leen tàggatoon lu ëpp
    • Amoon nañu ordinatër yu néew lool
    • Jëfandikoo nañu tokenizer bu jaarul yoon
    • Deñu leen tàggatoon bu baax ci seen dayo
  5. Ci gàttal, naka lañu xaymaa xayma ci jàngat bu Chinchilla bi?
    • Dafa méngoo ak limu GPU yi kese
    • Dayo bu amul benn njeexte ci dayo done
    • méngoo ak parametre yi jot tàggat jetons
    • Dafa méngoo ak dayo vokabileer bi
  6. Why might a deployment objective favor a smaller model trained on extra data rather than the training-compute optimum alone?
    • Repeated inference costs can matter alongside the initial training budget
    • Inference has no resource cost for any model
    • The Chinchilla study proved every scaling relationship false
    • More data always reduces both training and serving work to zero
  7. Under the approximation that training compute is proportional to N times D, how must D change if N doubles while that compute budget stays fixed?
    • D must also double
    • D must quadruple
    • D must remain unchanged
    • D must approximately halve
  8. Which three types of term appear in Chinchilla’s parametric loss fit?
    • Only GPU count, electricity price, and elapsed seconds
    • An irreducible base term, a model-size contribution, and a data-size contribution
    • Only vocabulary size, file size, and model name
    • Only a linear token count with no model-size term