Odds Ratio Preference Optimization Quiz

Answers and explanations appear as you take the quiz.

1. What is the main practical advantage of ORPO over methods like DPO?
   • It eliminates the need for a separate reference model
   • It requires no training data at all
   • It only works on image models
   • It guarantees zero hallucinations
2. What does the 'odds ratio' in ORPO compare?
   • Training speed versus accuracy
   • The odds of the chosen response versus the rejected response
   • Two different learning rates
   • The model size versus dataset size
3. What form does an ORPO preference example contain?
   • Only a single unlabeled response
   • A prompt with two identical completions
   • A prompt with one chosen response and one disfavored response
   • A reward score with no associated responses
4. How does ORPO convert a response probability p into odds?
   • One minus p divided by p
   • The square root of p
   • The logarithm of the prompt length
   • p divided by one minus p
5. The ORPO loss is best described as which combination?
   • SFT cross-entropy plus a weighted odds-ratio preference term
   • Only a reward-model score
   • A pure reinforcement-learning return
   • Random noise injection
6. Which response receives the supervised fine-tuning component of ORPO?
   • Only the rejected response
   • The chosen or preferred response
   • A response sampled after deployment
   • No response; ORPO has no language-modeling term
7. Which parameter controls the weight of ORPO’s preference term relative to the supervised loss?
   • The tokenizer vocabulary size
   • The number of attention heads
   • The preference-loss coefficient
   • The prompt’s character count
8. Why should an ORPO result report its preference data and evaluation setup?
   • ORPO results are identical across datasets
   • The objective removes the need for evaluation
   • The number of model layers determines every preference
   • Outcomes depend on the dataset, coefficient, base model, and evaluation