Bài kiểm tra có liên kết hướng dẫn · Cứng Cấp độ
Hỏi LLM họ tự tin như thế nào Câu đố
Kiểm tra hiệu chuẩn độ tin cậy, ECE, sự khác biệt trong nghiên cứu, khả năng mã thông báo, xác thực theo nhiệm vụ cụ thể và giới hạn câu trả lời lặp lại cho quy trình làm việc tiếp theo.
Câu hỏi 1 của 8
What does a model-generated “80% confident” statement establish by itself?
Xem trước mọi câu hỏi8 câu hỏi
Câu trả lời và giải thích xuất hiện khi bạn làm bài kiểm tra.
- What does a model-generated “80% confident” statement establish by itself?
- That the answer is correct 80% of the time
- Only that the model generated a confidence statement, not a calibrated probability
- That it used retrieved sources
- That it has a calibrated uncertainty module
- How should calibration of a confidence score be assessed?
- By asking the same question to one model twice
- By inspecting one answer
- Across a relevant labeled set, comparing confidence levels with observed correctness
- By checking whether the score is a round number
- What does the guide say about research on verbalized confidence?
- It is always better than token likelihoods
- It is universally calibrated on current systems
- Every study finds it useless
- Results vary by model, prompt, and evaluation data
- What does ECE summarize?
- Mismatch between confidence and observed accuracy across confidence bins
- The number of tokens in an answer
- The quality of a model’s explanations
- Whether each individual answer cites a source
- What do generated-token log probabilities directly describe?
- The probability the whole answer is factually correct
- Likelihood assigned to generated tokens, not directly correctness of the complete answer
- The user’s confidence
- Whether the answer is legally safe
- Before using confidence to route production tickets, what should a team do?
- Remove human review
- Set a threshold by intuition
- Validate score behavior on labeled examples similar to production cases
- Assume all categories calibrate equally
- Which limitation applies to repeated-answer agreement as a confidence signal?
- Nó không thể đo được
- It always reduces latency
- Samples may share the same misconception
- Agreement proves the answer is true
- In a consequential workflow, how should verbal confidence be used?
- As an unverified automatic approval rule
- Trust any answer above 50%
- Replace external evidence with the score
- As one validated signal alongside evidence and review