Process Reward Models Quiz

Answers and explanations appear as you take the quiz.

1. What primarily distinguishes a process reward model from an outcome reward model?
   • It scores each intermediate reasoning step, not just the final answer
   • It runs faster at inference time
   • It requires no training data
   • It only works on image inputs
2. Which well-known dataset of human step-level math labels is associated with PRM research?
   • ImageNet
   • PRM800K
   • SQuAD
   • WikiText
3. Why can an outcome-only reward signal be misleading?
   • Final answers are never correct
   • Outcome models cannot read text
   • A correct final answer can result from flawed reasoning, or a wrong answer from mostly-correct reasoning
   • It always overfits to grammar
4. A common way to turn per-step scores into a single chain score is to:
   • Add a random number to each step
   • Ignore all but the first step
   • Use only the token count
   • Take the minimum step probability across the chain
5. What does the Math-Shepherd approach use to label steps automatically?
   • Monte Carlo rollouts estimating how often a step leads to a correct answer
   • Manual expert review of every step
   • Translating steps into another language
   • Counting punctuation marks
6. Which risk is especially relevant when training models against a PRM?
   • The model forgets how to read
   • Reward hacking, where steps look good to the PRM without being genuinely correct
   • The PRM consumes no compute
   • Steps become impossible to generate
7. How can a process reward model guide a tree search over reasoning?
   • Expand every branch equally without scoring
   • Choose branches by their final token length
   • Prioritize expanding partial solutions whose intermediate steps receive stronger scores
   • Replace all branches with one randomly selected answer
8. What does the PRM800K dataset contain?
   • Only final answer labels for image captions
   • Audio recordings of spoken explanations
   • A corpus of code commits with no reasoning labels
   • Human labels for intermediate steps in mathematical reasoning solutions