Data Quality Monitoring in Production Quiz

Answers and explanations appear as you take the quiz.

1. What does a schema check detect in a live inference feed?
   • Unexpected field names, types or structures
   • A guaranteed drop in model accuracy
   • A causal change in user behavior
   • The model's calibration error
2. What does a freshness monitor detect?
   • A change in model weights
   • Data that have not been updated within the expected time
   • A label mismatch in a training batch
   • An unexpected prediction class
3. How should an unseen categorical value be handled?
   • Drop the model silently
   • Map it randomly to a known category
   • Apply an explicit unknown-category policy and monitor its rate
   • Assume the value is always invalid
4. Which mismatch is an example of train-serving skew?
   • A difference between two model architectures
   • A change in GPU memory usage
   • A delayed ground-truth label
   • A difference between how features are computed in training and inference
5. Why can an overly strict distribution threshold cause production issues?
   • It guarantees detection of every defect
   • Legitimate seasonal changes may trigger false alarms or blocks
   • It prevents all schema changes
   • It improves model calibration automatically
6. What does passing data-quality checks establish?
   • The feature relationships are unchanged
   • The model is fair across all groups
   • Inputs met the configured checks, not that predictions are accurate
   • The data are causally valid
7. Where might latency-sensitive data checks run?
   • In a dashboard without storing results
   • All validation only after labels arrive
   • Only in the model-training notebook
   • Critical schema checks inline, with heavier analysis asynchronously or in batches
8. What should a team do when a production data alert fires?
   • Trace the source and determine whether the change is expected before updating rules
   • Change the schema immediately to silence the alert
   • Retrain automatically without checking data
   • Delete the alert history