Guide-linked quiz · Hard Level

Data-Constrained Scaling Laws Quiz

Reason about limited unique data, repeated training tokens, epoch budgets, diminishing returns, held-out evaluation, and changes in data mixtures.

Question 1 of 8

A corpus contains ten billion unique tokens and is processed for three complete epochs. Which accounting is correct?

Preview every question8 questions

Answers and explanations appear as you take the quiz.

  1. A corpus contains ten billion unique tokens and is processed for three complete epochs. Which accounting is correct?
    • Ten billion unique tokens and thirty billion processed tokens
    • Thirty billion unique tokens and ten billion processed tokens
    • Three billion unique tokens and thirty billion processed tokens
    • Thirty billion independent documents
  2. Why can two runs with the same processed-token count learn differently?
    • Token counts determine quality exactly
    • One may use more unique information while the other repeats a smaller corpus
    • Every repeated token adds the same independent evidence as fresh data
    • Only the number of epochs can affect learning
  3. How should the cited study’s finding about roughly four epochs be interpreted?
    • Every model must stop after exactly four epochs
    • Four epochs always equal four times as much unique data
    • It is an empirical result under tested conditions, not a universal training rule
    • It proves that data quality never matters
  4. What does a fitted effective-data quantity represent in this setting?
    • The exact number of original documents
    • The storage size of the dataset
    • A guaranteed number of independent facts
    • A modeling quantity that can discount repeated exposures
  5. Training loss keeps falling while held-out loss rises after more repetition. Which response follows the guide?
    • Investigate memorization or mismatch before extending the run
    • Ignore held-out results because the training score improved
    • Add the held-out examples to training and keep the same evaluation claim
    • Assume more epochs must fix the problem
  6. Which preparation helps make a unique-token comparison meaningful across experiments?
    • Change tokenizers without documenting it
    • Record tokenization, filtering, and duplicate treatment
    • Report only the number of checkpoints
    • Count every generated copy as new independent data
  7. What should a team do when adding code or synthetic material to a constrained text corpus?
    • Assume any additional token improves every task
    • Skip provenance checks if total compute is unchanged
    • Evaluate the new mixture on relevant held-out tasks and inspect provenance and overlap
    • Treat all generated text as verified ground truth
  8. What limitation can remain even when a team trains a larger model on more repetitions?
    • The number of processed tokens must stay fixed
    • The original corpus automatically becomes larger
    • The held-out set becomes unnecessary
    • Missing domain coverage may still be missing

Keep testing yourself

More quizzes picked for your level.

View all quizzes