Retrieval Metrics: Recall@K, MRR and nDCG

Retrieval evaluation starts with queries, a document collection, and relevance judgments. A metric summarizes how a ranked result list compares with those judgments. Recall@K asks what fraction of relevant items appear within the first K results. It is useful when finding all relevant evidence matters, but it ignores the ordering among those retrieved items. Mean reciprocal rank (MRR) scores the rank of the first relevant result: a relevant item at rank one contributes 1, at rank two contributes one-half, and so on; query scores are averaged. This fits tasks where users mainly need one good result near the top. It does not reward additional relevant results after the first. Normalized discounted cumulative gain (nDCG) uses relevance grades and discounts items lower in the ranking. It can distinguish highly relevant from partially relevant documents and rewards placing stronger matches earlier. The exact gain and discount conventions should be documented when comparing implementations. These formulas summarize judged rankings and do not measure answer generation or user satisfaction. These measures depend on a test collection and relevance labels. Incomplete judgments can make a relevant but unjudged result look wrong, and query selection affects conclusions. For retrieval-augmented generation, report retrieval metrics separately from answer-level grounding, correctness, and usefulness. Choose metrics that match the product goal, inspect examples, and evaluate on representative queries rather than selecting whichever score looks largest.