Bài kiểm tra có liên kết hướng dẫn · Trung bình Cấp độ
Trắc nghiệm Ý nghĩa Thực sự của Điểm Tin cậy AI
Giải thích việc hiệu chuẩn, ý nghĩa ở cấp độ tổng thể, lựa chọn ngưỡng, kiểm tra nhóm phụ và tại sao điểm tin cậy của mô hình không phải là sự đảm bảo.
Câu hỏi 1 của 8
A calibrated binary classifier assigns 0.8 confidence to many cases. What should happen across that group over time?
Xem trước mọi câu hỏi8 câu hỏi
Câu trả lời và giải thích xuất hiện khi bạn làm bài kiểm tra.
- A calibrated binary classifier assigns 0.8 confidence to many cases. What should happen across that group over time?
- About 80 percent should be positive, across a sufficiently large representative group
- Every individual case must be correct
- The model must abstain on every case
- The score must equal accuracy on every subgroup
- What should you ask before interpreting a product's '90% confidence' badge?
- Whether the number uses a large font
- Whether the score is a calibrated probability and what outcome it predicts
- Whether the product has a dark theme
- Whether the model was written in Python
- Why should calibration data be separate from the model's training data?
- To make every output equal 0.5
- To remove the need for labels
- To reduce overly optimistic estimates from examples the model already fit
- To guarantee no future distribution shift
- A model is calibrated overall but not for one language group. What follows?
- The overall score is sufficient for every user
- The language group must be removed from evaluation
- Calibration is irrelevant
- The workflow should inspect subgroup outcomes and adjust use or calibration
- What does discrimination measure in this context?
- How well the model ranks cases with different outcomes
- Whether a score is displayed as a percentage
- Whether a user can change the threshold
- Mô hình chạy nhanh như thế nào
- When choosing an automation threshold, what should matter?
- The number of decimal places in the score
- The costs of false accepts and false rejects in the actual workflow
- The model's training language only
- Whether the threshold is a round number
- Why are token probabilities not automatically a factuality score for an entire generated answer?
- They cannot be stored
- They are always zero
- They measure token likelihoods, not a validated probability that the response is correct
- They only apply to images
- A low-confidence case could cause serious harm. Which workflow response is appropriate?
- Hide the score and auto-approve it
- Round the score upward
- Assume calibration makes the case correct
- Escalate or abstain according to a tested workflow