Feedback Loops in ML Systems Quiz

Answers and explanations appear as you take the quiz.

1. How can a recommender create a feedback loop?
   • Its exposure choices affect clicks that later become training evidence
   • Its predictions are never stored
   • Its model weights are fixed
   • Its users cannot respond to recommendations
2. Why may hiring outcomes be missing for applicants screened out by a model?
   • Their labels are always negative
   • They do not receive the downstream opportunity that generates an outcome
   • The model stores every outcome automatically
   • The feature schema removes labels
3. What does recording which items were shown help distinguish?
   • A GPU failure from an API timeout
   • Training data from model parameters
   • Exposure opportunity from user preference evidence
   • A label from a feature name
4. When can inverse-propensity evaluation be unreliable?
   • When the target policy matches the logging policy
   • When all propensities are recorded correctly
   • When randomized experiments have enough data
   • When the new policy chooses actions with no support in logged data
5. Why monitor label coverage by score or group?
   • Coverage proves the model is fair
   • Observation rates may differ because the model routes cases differently
   • Scores are always outcomes
   • Every group must have identical labels
6. Which data-collection strategy can reveal outcomes for less-shown alternatives?
   • Remove the need for logging
   • Guarantee every recommendation is relevant
   • Gather evidence about alternatives that a deterministic policy rarely shows
   • Make all users receive identical items
7. Why might more training data fail to remove feedback-loop bias?
   • Bias is always random
   • Larger datasets cannot contain labels
   • Training never uses user interactions
   • New data may be generated by the same selective policy
8. What kind of system property is a feedback loop?
   • A property of the deployed sociotechnical process and its data collection
   • Only a gradient calculation
   • A model architecture name
   • A feature scaling method