Minimum Bayes Risk Decoding Quiz

Answers and explanations appear as you take the quiz.

1. What does MBR decoding select instead of the single most probable sequence?
   • The candidate with the highest expected utility against other likely outputs
   • The shortest sequence
   • A random sample
   • The longest sequence
2. Since true references are unavailable, what does MBR use as pseudo-references?
   • A fixed external corpus
   • The pool of sampled candidate outputs themselves
   • Human annotations
   • The training set
3. Why is naive MBR computationally expensive?
   • It requires retraining the model
   • It needs a larger vocabulary
   • Pairwise comparison of N candidates is O(N squared) metric calls
   • It uses no GPU
4. What has COMET been used for in minimum Bayes risk machine-translation experiments?
   • As a universal guarantee of better translation quality
   • As a tokenizer for the candidate pool
   • As an exact reference translation available at inference
   • As a learned utility metric in evaluated translation settings
5. Self-consistency in reasoning relates to MBR because it:
   • Samples many chains and picks the most agreed-upon answer
   • Always takes the first sampled answer
   • Uses beam search internally
   • Ignores the model's distribution
6. How is risk generally expressed in minimum Bayes risk decision-making?
   • Always one minus a normalized similarity score
   • Expected loss under a chosen loss function and reference distribution
   • The model’s parameter count divided by accuracy
   • The number of candidates sampled
7. If the chosen utility metric disagrees with human judgments, what will MBR primarily optimize?
   • Human preference automatically overrides the metric
   • The model’s raw next-token likelihood
   • The expected utility under that selected metric
   • The shortest candidate in the pool
8. If a high-quality answer is missing from the sampled candidate pool, what can MBR do?
   • It guarantees the missing answer is generated
   • It changes the model weights to add the answer
   • It replaces the utility metric with a human reference
   • It can only select or rerank candidates that are in the pool