引導式測驗 · 中等 等級
評估多輪對話測驗
評估完整對話的背景、任務成功、復原和安全,同時檢查基準樣本和自動判斷限制。
問題 1 的 8
Why evaluate a conversation across multiple turns?
預覽每個問題8 問題
當您參加測驗時,就會出現答案和解釋。
- Why evaluate a conversation across multiple turns?
- Single-turn scores always measure long-term memory
- Multi-turn tests remove all uncertainty
- Only the last message matters
- A later response can fail to use earlier context or complete the overall goal
- Which dimension may matter in a multi-turn support dialogue?
- Only the first token latency
- Whether the assistant retained earlier facts and resolved the user’s issue
- Only the number of total turns
- The chatbot’s training loss
- What was the scope of MT-Bench-101 cited in the guide?
- A million production conversations from every industry
- Only single-turn math prompts
- 1,388 dialogues across 13 tasks, comprising 4,208 turns
- One conversation judged by one person
- Does a benchmark score guarantee performance in every deployed conversation?
- No; results apply to the benchmark tasks and sample
- Yes, if the benchmark has more than 1,000 turns
- Yes, if an automated judge was used
- No, because benchmarks provide no information
- Why validate an automated conversation judge against human review?
- To guarantee every human judge agrees
- To remove the need for test cases
- To make dialogue length irrelevant
- To check agreement, rubric quality, and possible judge bias
- What should a dialogue test do when a user corrects an earlier instruction?
- Ignore the correction and score only the first response
- End the test immediately
- Check whether the assistant incorporates the correction in later turns
- Treat the correction as a tool result
- Which cases can expose dialogue-level weaknesses?
- Only short greetings
- Long histories, interruptions, tool failures, and safety-sensitive turns
- Only repeated identical prompts
- A model card without conversations
- How should a team report multi-turn evaluation?
- Report sample selection, scoring rules, and failure categories
- Publish only a single average score
- Hide judge disagreement
- Report only answer length