Bradley-Terry Reward Modeling Quiz

Answers and explanations appear as you take the quiz.

1. What does the Bradley-Terry model convert into numeric scores?
   • Pairwise comparisons of which item is preferred
   • Absolute 1-to-10 ratings
   • Raw pixel values
   • Time-series measurements
2. In Bradley-Terry, the probability that A beats B is a function of what?
   • The product of their scores
   • The logistic of the difference between their scores
   • The ratio of their token counts
   • A fixed 50 percent regardless of scores
3. Why are Bradley-Terry rewards identifiable only up to an additive constant?
   • Because scores must sum to one
   • Because humans cannot count
   • Because the loss depends only on score differences
   • Because the sigmoid removes all information
4. What is the standard loss for a single preference comparison in reward modeling?
   • Mean squared error against a 1-to-10 label
   • Cross-entropy over the full vocabulary
   • Absolute difference of token lengths
   • Negative log-sigmoid of (reward of chosen minus reward of rejected)
5. Which method skips a separate reward model by optimizing the Bradley-Terry objective directly on the policy?
   • Direct Preference Optimization (DPO)
   • Proximal Policy Optimization (PPO)
   • Batch normalization
   • Gradient clipping
6. What is a key limitation of the Bradley-Terry assumption?
   • It requires GPUs to compute
   • It assumes a single consistent, transitive ranking that can break when preferences cycle
   • It only works on images
   • It cannot handle more than three items ever
7. Why can pairwise judgments be easier to collect than absolute quality scores?
   • Pairwise choices never contain disagreement
   • Absolute scores cannot be used for training
   • A rater can compare two candidates directly without calibrating a numeric scale
   • A pairwise label reveals every feature of the reward model
8. Which familiar ranking system is mathematically related to Bradley-Terry-style pairwise scoring?
   • PageRank for web pages
   • TF-IDF for document terms
   • A Fourier transform for audio
   • Elo ratings