NDCG and Ranking Metrics Quiz

Answers and explanations appear as you take the quiz.

1. Two relevant items appear in the top five. What is precision@5?
   • 2/5
   • 2 divided by all relevant candidates
   • 5/2
   • The rank of the first relevant item
2. Two of three relevant candidates appear in the top five. What is recall@5?
   • 2/5
   • 2/3
   • 3/2
   • 5/3
3. Which metric emphasizes the rank of the first relevant result?
   • Precision@k
   • Recall@k
   • MRR
   • NDCG only at the last position
4. What does the logarithmic discount in DCG do?
   • Normalizes by the number of queries
   • Increases credit as rank moves down the list
   • Removes relevance grades
   • Reduces credit for relevant items at lower ranks
5. How does ideal DCG contribute to normalized DCG?
   • Convert graded labels into binary labels
   • Scale ranking quality relative to the best ordering for that query
   • Measure the fraction of candidates retrieved
   • Find the first relevant rank only
6. Which task best matches MRR?
   • Measuring only class calibration
   • Retrieving every relevant candidate equally
   • Finding the first useful result as high as possible
   • Ranking all results by predicted probability but ignoring position
7. What can NDCG represent that binary precision@k may not?
   • The first relevant result rank only
   • The exact probability a user is satisfied
   • The fraction of all relevant candidates found
   • Graded relevance combined with position discount
8. Why report cutoff and relevance conventions with a ranking score?
   • They affect the metric definition and resulting value
   • They are required only for regression metrics
   • They never change NDCG
   • They reveal whether candidate items were causal