AI Early Warning Scores for Patient Deterioration Quiz

Answers and explanations appear as you take the quiz.

1. Which of these is one of the parameters scored in NEWS2?
   • Serum creatinine
   • Use of supplemental oxygen
   • White blood cell count
   • Blood lactate
2. What was a defining feature of Kaiser Permanente's Advance Alert Monitor deployment?
   • Alerts went straight to patients' phones
   • The model ran only in ICUs
   • Trained remote nurses screened alerts before they reached the bedside team
   • Alerts automatically ordered antibiotics
3. Why does the guide warn that a high AUROC does not guarantee better care?
   • What matters is how often an alert is right at the chosen threshold, the number of alerts per shift, lead time and whether the response changes treatment
   • AUROC only works for imaging models
   • AUROC is always lower for AI than for NEWS2
   • AUROC measures cost, not accuracy
4. How can orders for lactate or blood cultures cause a deterioration model to overstate its usefulness?
   • They slow down the EHR
   • They are recorded in the wrong units
   • They reset the model's score to zero
   • They signal that clinicians already suspect trouble, so the model may flag deterioration staff have already recognized
5. Why does computing AUROC over every observation inflate a deterioration model's performance?
   • Stable patients contribute hundreds of easy negative observations
   • It double-counts deaths
   • It ignores vital signs
   • It only includes ICU patients
6. What is the purpose of running a deterioration model in silent mode before go-live?
   • To hide the model from regulators
   • To reduce electricity use
   • To compute scores without showing them, so the hospital can measure local calibration, subgroup performance and alert volume
   • To train nurses without any patient data
7. If a deterioration model's positive predictive value at the alert threshold is 0.1, what is the number needed to evaluate?
   • 1
   • 10
   • 0.1
   • 100
8. What did the University of Michigan's 2021 independent evaluation of Epic's sepsis model show?
   • It performed exactly as the vendor reported
   • It worked only for children
   • It could replace blood cultures
   • It performed substantially worse than the developer had reported