Guide-linked quiz · Hard Level
Data-Constrained Scaling Laws Quiz
Reason about limited unique data, repeated training tokens, epoch budgets, diminishing returns, held-out evaluation, and changes in data mixtures.
Question 1 of 8
A corpus contains ten billion unique tokens and is processed for three complete epochs. Which accounting is correct?
Preview every question8 questions
Answers and explanations appear as you take the quiz.
- A corpus contains ten billion unique tokens and is processed for three complete epochs. Which accounting is correct?
- Ten billion unique tokens and thirty billion processed tokens
- Thirty billion unique tokens and ten billion processed tokens
- Three billion unique tokens and thirty billion processed tokens
- Thirty billion independent documents
- Why can two runs with the same processed-token count learn differently?
- Token counts determine quality exactly
- One may use more unique information while the other repeats a smaller corpus
- Every repeated token adds the same independent evidence as fresh data
- Only the number of epochs can affect learning
- How should the cited study’s finding about roughly four epochs be interpreted?
- Every model must stop after exactly four epochs
- Four epochs always equal four times as much unique data
- It is an empirical result under tested conditions, not a universal training rule
- It proves that data quality never matters
- What does a fitted effective-data quantity represent in this setting?
- The exact number of original documents
- The storage size of the dataset
- A guaranteed number of independent facts
- A modeling quantity that can discount repeated exposures
- Training loss keeps falling while held-out loss rises after more repetition. Which response follows the guide?
- Investigate memorization or mismatch before extending the run
- Ignore held-out results because the training score improved
- Add the held-out examples to training and keep the same evaluation claim
- Assume more epochs must fix the problem
- Which preparation helps make a unique-token comparison meaningful across experiments?
- Change tokenizers without documenting it
- Record tokenization, filtering, and duplicate treatment
- Report only the number of checkpoints
- Count every generated copy as new independent data
- What should a team do when adding code or synthetic material to a constrained text corpus?
- Assume any additional token improves every task
- Skip provenance checks if total compute is unchanged
- Evaluate the new mixture on relevant held-out tasks and inspect provenance and overlap
- Treat all generated text as verified ground truth
- What limitation can remain even when a team trains a larger model on more repetitions?
- The number of processed tokens must stay fixed
- The original corpus automatically becomes larger
- The held-out set becomes unnecessary
- Missing domain coverage may still be missing
Keep testing yourself
More quizzes picked for your level.