引導式測驗 · 硬 等級

數據受限縮放定律測驗

解釋有限的獨特資料、重複的訓練代幣、紀元預算、收益遞減、保留評估和資料混合的變化。

相關引導路徑數據受限縮放定律
問題 1 的 8

A corpus contains ten billion unique tokens and is processed for three complete epochs. Which accounting is correct?

預覽每個問題8 問題

當您參加測驗時,就會出現答案和解釋。

  1. A corpus contains ten billion unique tokens and is processed for three complete epochs. Which accounting is correct?
    • Ten billion unique tokens and thirty billion processed tokens
    • Thirty billion unique tokens and ten billion processed tokens
    • Three billion unique tokens and thirty billion processed tokens
    • Thirty billion independent documents
  2. Why can two runs with the same processed-token count learn differently?
    • Token counts determine quality exactly
    • One may use more unique information while the other repeats a smaller corpus
    • Every repeated token adds the same independent evidence as fresh data
    • Only the number of epochs can affect learning
  3. How should the cited study’s finding about roughly four epochs be interpreted?
    • Every model must stop after exactly four epochs
    • Four epochs always equal four times as much unique data
    • It is an empirical result under tested conditions, not a universal training rule
    • It proves that data quality never matters
  4. What does a fitted effective-data quantity represent in this setting?
    • The exact number of original documents
    • The storage size of the dataset
    • A guaranteed number of independent facts
    • A modeling quantity that can discount repeated exposures
  5. Training loss keeps falling while held-out loss rises after more repetition. Which response follows the guide?
    • Investigate memorization or mismatch before extending the run
    • Ignore held-out results because the training score improved
    • Add the held-out examples to training and keep the same evaluation claim
    • Assume more epochs must fix the problem
  6. Which preparation helps make a unique-token comparison meaningful across experiments?
    • Change tokenizers without documenting it
    • Record tokenization, filtering, and duplicate treatment
    • Report only the number of checkpoints
    • Count every generated copy as new independent data
  7. What should a team do when adding code or synthetic material to a constrained text corpus?
    • Assume any additional token improves every task
    • Skip provenance checks if total compute is unchanged
    • Evaluate the new mixture on relevant held-out tasks and inspect provenance and overlap
    • Treat all generated text as verified ground truth
  8. What limitation can remain even when a team trains a larger model on more repetitions?
    • The number of processed tokens must stay fixed
    • The original corpus automatically becomes larger
    • The held-out set becomes unnecessary
    • Missing domain coverage may still be missing