引導式測驗 · 硬 等級
配對和 Elo 評估測驗
區分成對偏好與絕對分數、Elo 更新與 Bradley-Terry 估計以及排名與通用品質聲明。
問題 1 的 8
Which limitation can pairwise evaluation reduce compared with an absolute 1-to-5 score?
預覽每個問題8 問題
當您參加測驗時,就會出現答案和解釋。
- Which limitation can pairwise evaluation reduce compared with an absolute 1-to-5 score?
- It can reduce disagreements caused by different uses of a numeric scale
- It removes all judge bias
- 它保證了事實的準確性
- It eliminates the need to sample prompts
- What real-world platform is cited as using pairwise comparisons at scale for model evaluation?
- A stock trading exchange
- Chatbot Arena
- A weather forecasting service
- A video streaming platform
- Where does the Elo rating system used for LLM comparisons originally come from?
- Credit scoring
- Search engine ranking
- Chess rating
- 天气预报模型
- What does the Bradley-Terry model estimate from pairwise outcomes?
- A universal absolute quality score
- The number of tokens in each response
- A guarantee that every voter agrees
- A latent strength parameter related to the probability of one item beating another
- Why is randomizing the order of outputs important when using an LLM-as-judge for pairwise comparisons?
- It controls for position bias, where a judge might favor whichever output is shown first
- It makes the comparison run faster
- It reduces the API cost of the comparison
- It is required by MCP protocol rules
- According to the deep dive, what should be treated cautiously about Elo ratings from very few comparisons?
- That they are always higher than they should be
- That they carry real uncertainty and shouldn't be treated as precise until enough comparisons accumulate
- That they can only be computed for exactly two competitors
- That they cannot be visualized in a leaderboard
- In an Elo update, what determines how much a rating changes after a single match?
- The alphabetical order of the competitors' names
- The total number of competitors in the leaderboard
- The difference between the actual outcome and the expected outcome, scaled by a constant K
- The length of the input prompt
- Why might a prompt engineer switch from 1-to-5 absolute scoring to pairwise comparison mid-project?
- Because pairwise comparisons are always cheaper to run
- Because absolute scoring is not supported by any tool
- Because pairwise comparisons require no judge at all
- Because absolute scores clustered near the top of the scale, giving little separation between prompt versions