引導式測驗 · 中等 等級
監控無真實標籤模型測驗
將代理監控與已驗證的效能區分開來,並處理延遲標籤、審查和選擇性結果收集。
問題 1 的 8
What can a prediction-distribution shift establish without labels?
預覽每個問題8 問題
當您參加測驗時,就會出現答案和解釋。
- What can a prediction-distribution shift establish without labels?
- The model's inputs or outputs changed under the monitored data
- That accuracy definitely declined
- That the new predictions are all wrong
- That the labels are unbiased
- Why can model confidence not serve as ground truth by itself?
- Confidence values are always missing
- A model can be confidently wrong or poorly calibrated
- Labels arrive instantly
- Confidence measures only latency
- What does a delayed-label maturity window help determine?
- Whether a feature should be standardized
- Which model architecture is most accurate
- When an outcome has had enough time to be observed
- How many requests were served
- What bias can arise if only appealed cases receive labels?
- The label delay disappears
- All predictions become positive
- The model is necessarily calibrated
- The labeled sample may not represent all predictions
- Why retain a stable request ID and model version at prediction time?
- To change the prediction after the fact
- To join a later outcome to the exact prediction record
- To infer labels from confidence
- To remove privacy obligations
- Which condition describes right-censoring in delayed outcome monitoring?
- The model has no input features
- The prediction is hidden from users
- The outcome has not yet had enough time to occur or be observed
- A label is stored twice
- How should a human-review proxy be interpreted?
- As a replacement for delayed labels
- As exact model accuracy
- As proof every unreviewed case is correct
- As an indicator that may also reflect reviewer policy and case mix
- What should happen when verified labels eventually arrive?
- Evaluate actual outcomes and revisit earlier proxy-based conclusions
- Keep proxies as the only metric
- Discard all prediction logs
- Assume the proxies were perfectly calibrated