Test-Time Compute Scaling Quiz

Answers and explanations appear as you take the quiz.

1. What does 'test-time compute' refer to?
   • Computation spent training the model on data
   • The time it takes to download a model
   • Computation spent when the model answers a query at inference
   • The electricity used to cool data centers
2. In test-time scaling studies, what can extra inference compute be used for?
   • Changing the model’s training data during the request
   • Increasing the number of model parameters automatically
   • Guaranteeing correctness without evaluation
   • Searching or revising candidate solutions and checking their quality
3. What is the core trade-off of test-time compute scaling?
   • Higher accuracy in exchange for more latency and cost
   • Less memory but slower training
   • More data but smaller models
   • Better graphics for lower resolution
4. What is 'best-of-N' sampling?
   • Training on the N largest datasets
   • Generating N candidate answers and selecting the best one
   • Using only the first N words of a prompt
   • Running the model on N different computers for speed
5. Why is test-time compute considered a new 'scaling axis'?
   • It replaces the need for any training
   • It only works on small models
   • It improves performance by adding inference compute rather than only growing model size
   • It reduces the number of users a model can serve
6. What surprising result does test-time compute enable?
   • Models can run without any GPUs
   • Training data becomes unnecessary
   • Models always answer faster
   • A smaller model thinking longer can rival a much larger model answering instantly
7. What is one reason a fixed best-of-N strategy may waste inference compute?
   • It spends the same candidate budget on easy and difficult prompts
   • It modifies model weights on every query
   • It cannot generate multiple candidates
   • It eliminates the need for scoring
8. What did the test-time compute paper compare with best-of-N sampling?
   • A strategy that disables inference
   • A compute-optimal strategy that allocates effort based on expected solution quality
   • A larger tokenizer vocabulary
   • A method that changes image resolution