Chinchilla Compute-Optimal Training Quiz

Answers and explanations appear as you take the quiz.

1. How many language models did the Chinchilla study train for its scaling analysis?
   • Over 400
   • Fewer than 10
   • Exactly 20
   • More than 100,000
2. What parameter-count range did the paper’s model sweep cover?
   • 1 million to 10 million
   • About 70 million to over 16 billion parameters
   • 70 billion to over 1 trillion
   • Only exactly 70 billion
3. What training-token range did the paper’s sweep cover?
   • 5 million to 50 million
   • Only 300 billion tokens
   • About 5 billion to 500 billion tokens
   • 1 trillion to 10 trillion tokens
4. What did the paper’s first estimation approach vary while holding model sizes fixed?
   • The tokenizer for every step
   • The number of evaluation benchmarks only
   • The dataset language while keeping training curves unused
   • Training duration, measured by the number of training sequences or tokens
5. In an IsoFLOP profile, what is held fixed while model sizes are compared?
   • The total training compute budget in FLOPs
   • The token count for every model regardless of size
   • The number of evaluation raters
   • The vocabulary size only
6. What shape did the paper look for in loss versus model size on an IsoFLOP profile?
   • A line that always decreases with no minimum
   • A valley with a minimum-loss model size for that compute budget
   • A constant horizontal score for every size
   • A random pattern unrelated to compute
7. How did the paper fit the parameters of its third, parametric loss approach?
   • By hand-labeling every token
   • Using K-means without a loss function
   • With Huber loss and L-BFGS, selecting a good fit from a grid of initializations
   • By comparing only model names
8. Which training dataset did the Chinchilla model use?
   • ImageNet only
   • The Common Voice speech corpus
   • A dataset created only from generated model answers
   • MassiveText, with a slightly different subset distribution from Gopher