गाइड से जुड़ी प्रश्नोत्तरी · मध्यम स्तर
एक चिकित्सक प्रश्नोत्तरी के रूप में मेडिकल एआई टूल्स का मूल्यांकन करना
विक्रेता सत्यापन डेटा का निर्धारण करने वाले परीक्षण, सकारात्मक पूर्वानुमानित मूल्य और अंशांकन स्थानीय स्तर पर क्यों मायने रखते हैं, और एफडीए 510(के) निकासी क्या दिखाती है।
प्रश्न 1 का 8
A vendor reports an AUC of 0.85 for its deterioration model. What does the guide say this number fails to tell a hospital?
प्रत्येक प्रश्न का पूर्वावलोकन करें8 प्रश्न
जैसे ही आप प्रश्नोत्तरी लेते हैं उत्तर और स्पष्टीकरण दिखाई देते हैं।
- A vendor reports an AUC of 0.85 for its deterioration model. What does the guide say this number fails to tell a hospital?
- How well the model ranks cases in the vendor's own dataset
- Whether the model was built with machine learning
- How the tool will behave with the hospital's own patients, workflow and alert threshold
- Whether the model can be installed in an EHR
- What did the 2021 external validation of a proprietary sepsis model, published in JAMA Internal Medicine, find?
- Much weaker discrimination than the developer reported, with many missed cases and frequent alerts
- Performance identical to the developer's figures
- Better performance at the external site than at the development site
- That the FDA had revoked the model's clearance
- A model was validated where a condition affects 10 percent of patients. Why might it generate more false alarms on a unit where only 2 percent have the condition?
- Calibration always improves at lower prevalence
- AUC automatically falls when prevalence falls
- Sensitivity rises sharply at lower prevalence
- Positive predictive value depends on prevalence, so a larger share of alerts are false when the condition is rarer
- According to the guide, which pathway do most AI medical devices use to reach the US market?
- Premarket approval backed by randomized trials
- The 510(k) pathway, based on substantial equivalence to an existing device
- No FDA review of any kind
- Emergency use authorization
- What is a silent or shadow deployment of a clinical AI model?
- Muting the sound on alert notifications
- A vendor updating the model without telling the customer
- Letting the model score live patients without showing results to clinicians, so local performance can be measured
- Running the model only during overnight shifts
- A dermatology group learns that darker skin tones were rare in a classifier's training images. Which evaluation area does this concern?
- Post-deployment monitoring
- Bias, meaning performance across patient subgroups
- Regulatory status
- Alert threshold selection
- A model assigns 30 percent risk to a group of patients, but only 10 percent of them go on to have the outcome. What problem does this show?
- A low AUC, necessarily
- Substantial equivalence failure
- High positive predictive value
- Poor calibration
- After go-live, a hospital switches to a new lab assay and the values feeding the model shift. What does the guide call this, and how should it be handled?
- Input drift, which ongoing monitoring should detect
- Algorithm aversion, which training should fix
- Substantial equivalence, which the FDA handles
- Prevalence shift, which needs no action