引導式測驗 · 中等 等級
龍貓縮放定律測驗
測試 Chinchilla 訓練預算結果、70B/1.4T 範例、縮放比例、近似計算以及與部署優化的差異。
問題 1 的 8
Chinchilla 發現參數和令牌應該以相似的比例縮放意味著什麼?
預覽每個問題8 問題
當您參加測驗時,就會出現答案和解釋。
- Chinchilla 發現參數和令牌應該以相似的比例縮放意味著什麼?
- Both should grow with comparable scaling exponents under the studied training-compute objective
- The numerical token count must equal the parameter count
- Training data does not affect the fitted loss
- Every deployment must choose the largest possible model
- 70B 參數、1.4T 代幣 Chinchilla 實驗說明了什麼代幣/參數比率?
- Exactly one token per parameter
- About two thousand tokens per parameter
- About twenty training tokens per parameter
- A ratio determined only by the number of GPUs
- 儘管龍貓體型小得多,但哪種型號的性能優於哪款?
- GPT-2
- 地鼠
- 伯特
- 曼巴
- Chinchilla 當時對 GPT-3 這樣的模型意味著什麼?
- 他們訓練過度
- 他們可用的計算太少
- 他們使用了錯誤的標記器
- 相對於他們的體型而言,他們的訓練不足
- Chinchilla 分析中的計算大致是如何進行的?
- 僅與 GPU 數量成正比
- 與資料大小無關
- 與參數乘以訓練令牌成正比
- 與詞彙量成正比
- Why might a deployment objective favor a smaller model trained on extra data rather than the training-compute optimum alone?
- Repeated inference costs can matter alongside the initial training budget
- Inference has no resource cost for any model
- The Chinchilla study proved every scaling relationship false
- More data always reduces both training and serving work to zero
- Under the approximation that training compute is proportional to N times D, how must D change if N doubles while that compute budget stays fixed?
- D must also double
- D must quadruple
- D must remain unchanged
- D must approximately halve
- Which three types of term appear in Chinchilla’s parametric loss fit?
- Only GPU count, electricity price, and elapsed seconds
- An irreducible base term, a model-size contribution, and a data-size contribution
- Only vocabulary size, file size, and model name
- Only a linear token count with no model-size term