Викторина, связанная с руководством · Жесткий Уровень
Викторина для парной оценки и оценки Эло
Отличайте парное предпочтение от абсолютных оценок, обновления Эло от оценок Брэдли-Терри и рейтинги от универсальных заявлений о качестве.
Вопрос 1 из 8
Which limitation can pairwise evaluation reduce compared with an absolute 1-to-5 score?
Предварительный просмотр каждого вопроса8 вопросы
Ответы и пояснения появляются по мере прохождения теста.
- Which limitation can pairwise evaluation reduce compared with an absolute 1-to-5 score?
- It can reduce disagreements caused by different uses of a numeric scale
- It removes all judge bias
- Это гарантирует фактическую точность
- It eliminates the need to sample prompts
- What real-world platform is cited as using pairwise comparisons at scale for model evaluation?
- A stock trading exchange
- Chatbot Arena
- A weather forecasting service
- A video streaming platform
- Where does the Elo rating system used for LLM comparisons originally come from?
- Credit scoring
- Search engine ranking
- Chess rating
- Модели прогнозирования погоды
- What does the Bradley-Terry model estimate from pairwise outcomes?
- A universal absolute quality score
- The number of tokens in each response
- A guarantee that every voter agrees
- A latent strength parameter related to the probability of one item beating another
- Why is randomizing the order of outputs important when using an LLM-as-judge for pairwise comparisons?
- It controls for position bias, where a judge might favor whichever output is shown first
- It makes the comparison run faster
- It reduces the API cost of the comparison
- It is required by MCP protocol rules
- According to the deep dive, what should be treated cautiously about Elo ratings from very few comparisons?
- That they are always higher than they should be
- That they carry real uncertainty and shouldn't be treated as precise until enough comparisons accumulate
- That they can only be computed for exactly two competitors
- That they cannot be visualized in a leaderboard
- In an Elo update, what determines how much a rating changes after a single match?
- The alphabetical order of the competitors' names
- The total number of competitors in the leaderboard
- The difference between the actual outcome and the expected outcome, scaled by a constant K
- The length of the input prompt
- Why might a prompt engineer switch from 1-to-5 absolute scoring to pairwise comparison mid-project?
- Because pairwise comparisons are always cheaper to run
- Because absolute scoring is not supported by any tool
- Because pairwise comparisons require no judge at all
- Because absolute scores clustered near the top of the scale, giving little separation between prompt versions