引導式測驗 · 中等 等級

龍貓縮放定律測驗

測試 Chinchilla 訓練預算結果、70B/1.4T 範例、縮放比例、近似計算以及與部署優化的差異。

相關引導路徑龍貓縮放法則
問題 1 的 8

Chinchilla 發現參數和令牌應該以相似的比例縮放意味著什麼?

預覽每個問題8 問題

當您參加測驗時,就會出現答案和解釋。

  1. Chinchilla 發現參數和令牌應該以相似的比例縮放意味著什麼?
    • Both should grow with comparable scaling exponents under the studied training-compute objective
    • The numerical token count must equal the parameter count
    • Training data does not affect the fitted loss
    • Every deployment must choose the largest possible model
  2. 70B 參數、1.4T 代幣 Chinchilla 實驗說明了什麼代幣/參數比率?
    • Exactly one token per parameter
    • About two thousand tokens per parameter
    • About twenty training tokens per parameter
    • A ratio determined only by the number of GPUs
  3. 儘管龍貓體型小得多,但哪種型號的性能優於哪款?
    • GPT-2
    • 地鼠
    • 伯特
    • 曼巴
  4. Chinchilla 當時對 GPT-3 這樣的模型意味著什麼?
    • 他們訓練過度
    • 他們可用的計算太少
    • 他們使用了錯誤的標記器
    • 相對於他們的體型而言,他們的訓練不足
  5. Chinchilla 分析中的計算大致是如何進行的?
    • 僅與 GPU 數量成正比
    • 與資料大小無關
    • 與參數乘以訓練令牌成正比
    • 與詞彙量成正比
  6. Why might a deployment objective favor a smaller model trained on extra data rather than the training-compute optimum alone?
    • Repeated inference costs can matter alongside the initial training budget
    • Inference has no resource cost for any model
    • The Chinchilla study proved every scaling relationship false
    • More data always reduces both training and serving work to zero
  7. Under the approximation that training compute is proportional to N times D, how must D change if N doubles while that compute budget stays fixed?
    • D must also double
    • D must quadruple
    • D must remain unchanged
    • D must approximately halve
  8. Which three types of term appear in Chinchilla’s parametric loss fit?
    • Only GPU count, electricity price, and elapsed seconds
    • An irreducible base term, a model-size contribution, and a data-size contribution
    • Only vocabulary size, file size, and model name
    • Only a linear token count with no model-size term