AI Model Monitoring Quiz

Answers and explanations appear as you take the quiz.

1. A feature’s input distribution shifts after a product change. What does this establish?
   • That the model must be retrained immediately
   • That the new data are invalid
   • A reason to investigate, not automatic proof that accuracy declined
   • That task quality improved
2. Why track missing fields and invalid ranges before outcome labels arrive?
   • They measure accuracy directly
   • They replace delayed outcome evaluation
   • They prove the model is biased
   • They can expose pipeline problems before predictive quality can be measured
3. An unlabeled drift metric rises. What is still needed to establish task performance?
   • Reliable outcome-based evaluation
   • A larger dashboard
   • A second drift metric
   • A new prompt template
4. Why inspect label collection delays and selection?
   • Labels always arrive immediately
   • Delayed or selective labels can make performance evidence look more complete than it is
   • Only model outputs need versioning
   • A delayed label is automatically a bad outcome
5. A feature definition changed and the model’s inputs shifted. What should happen before retraining?
   • Assume all drift requires retraining
   • Ignore the shift if latency is stable
   • Investigate the cause and determine whether the shift is a data or software defect
   • Change the alert threshold until it disappears
6. What should a monitoring alert process specify?
   • Only a chart color
   • The model’s preferred action
   • A guarantee that drift will not occur
   • Thresholds, review ownership, and the recovery decision
7. After retraining, how should the intervention be checked?
   • Evaluate on appropriate material and continue measuring afterward
   • Compare only training accuracy
   • Assume retraining fixed the cause
   • Stop monitoring the old and new versions
8. Why include subgroup results and sample sizes in monitoring reports?
   • Averages already provide complete subgroup information
   • They show which populations the reported performance actually covers
   • Sample sizes never affect interpretation
   • Subgroups can be inferred from the model score