Brier Score Quiz

Answers and explanations appear as you take the quiz.

1. A parcel has a 70% chance of being late and is late. Which binary Brier loss follows from the guide's formula?
   • 0.30
   • 0.09
   • 0.49
   • 0.70
2. For the two parcels with losses of 0.04 and 0.09, which calculation produces their combined Brier score?
   • Add the losses to obtain 0.13
   • Subtract the losses to obtain 0.05
   • Take the larger loss, 0.09
   • Average the losses to obtain 0.065
3. A delivery model reports probabilities of being late, but its outcome column uses 1 for on-time delivery. What should the evaluator fix first?
   • Make the probability and outcome refer to the same event
   • Round all probabilities to whole numbers
   • Use only parcels predicted to be late
   • Replace the mean with the largest loss
4. Why can a model's Brier score improve without its calibration improving?
   • The score ignores predicted probabilities
   • The score always rewards higher confidence
   • Better separation of risk groups can also improve the score
   • Calibration affects only multiclass predictions
5. When escalations are rare, which comparison makes a ticket model's low Brier score more informative?
   • Comparing it with a model evaluated on a different event rate
   • Comparing it with a constant-rate baseline on the same tickets
   • Comparing only tickets whose predicted probability exceeds 0.5
   • Comparing only the models' percentages of correct labels
6. Which part of the classical Brier decomposition depends on the overall event rate?
   • Reliability
   • Resolution
   • The difference between reliability and resolution
   • Uncertainty
7. Why should an evaluator retain 0.6 and 0.9 instead of converting both delivery forecasts into the label 'late' before scoring?
   • Rounding removes the confidence information the Brier score evaluates
   • Thresholding preserves all probability information needed by the score
   • Rounding makes the score a direct measure of probability calibration
   • Only forecasts above the threshold need to be retained
8. Two software reports use different binary and multiclass Brier conventions. Which detail must be checked before their numbers are compared?
   • Whether both reports use the same classification threshold
   • Whether both models have the same accuracy
   • The score normalization and ordering of classes
   • Whether each class receives the same number of predictions