技術指南

檢索指標:Recall@K、MRR 和 nDCG

Recall@K measures how many known relevant items appear in the top K results, MRR emphasizes the rank of the first relevant item, and nDCG evaluates ranking quality using graded relevance with position discounts.

聆聽此指南AI 語音敘述 · ~3 min

本頁閱讀時間3分鐘
  1. 概述
  2. 深入探討
  3. 現實世界的實施
  4. The Future of Retrieval Metrics: Recall@K, MRR and nDCG
  5. 付諸實踐
  6. 不斷探索
  7. 常見問題

概述

Each metric reflects different judgments and use cases, and none directly measures whether a generated answer is correct.

深入探討

Retrieval evaluation starts with queries, a document collection, and relevance judgments. A metric summarizes how a ranked result list compares with those judgments. Recall@K asks what fraction of relevant items appear within the first K results. It is useful when finding all relevant evidence matters, but it ignores the ordering among those retrieved items. Mean reciprocal rank (MRR) scores the rank of the first relevant result: a relevant item at rank one contributes 1, at rank two contributes one-half, and so on; query scores are averaged. This fits tasks where users mainly need one good result near the top. It does not reward additional relevant results after the first. Normalized discounted cumulative gain (nDCG) uses relevance grades and discounts items lower in the ranking. It can distinguish highly relevant from partially relevant documents and rewards placing stronger matches earlier. The exact gain and discount conventions should be documented when comparing implementations. These formulas summarize judged rankings and do not measure answer generation or user satisfaction. These measures depend on a test collection and relevance labels. Incomplete judgments can make a relevant but unjudged result look wrong, and query selection affects conclusions. For retrieval-augmented generation, report retrieval metrics separately from answer-level grounding, correctness, and usefulness. Choose metrics that match the product goal, inspect examples, and evaluate on representative queries rather than selecting whichever score looks largest.

現實世界的實施

A legal search team uses Recall@20 when it needs to surface most known relevant passages for review.

A help center tracks MRR when users need one useful answer near the top.

A recommender uses graded relevance labels and nDCG to favor stronger matches earlier.

A RAG evaluation reports retrieval recall separately from whether the generated answer is supported.

The Future of Retrieval Metrics: Recall@K, MRR and nDCG

Retrieval systems will increasingly combine lexical, vector, hybrid, and reranking stages, making stage-specific evaluation important. Better relevance judgments and task-aligned graded labels can make nDCG more useful, while user studies can test whether rank gains improve outcomes. RAG evaluation will continue to pair retrieval metrics with answer-level checks. Future benchmarks should publish query sets, judgments, metric settings, and limitations to support meaningful comparisons. Teams should revisit judgments as collections and user needs change over time for all relevant users consistently.

付諸實踐

為什麼這很重要

  • 成本與預算. 多年來,架構決策決定著效能和營運成本。
  • 更明確的決策. 技術教育幫助團隊選擇正確的堆疊,而不僅僅是最新的堆疊。
  • 品質管控. 更好的工程選擇可以減少生產中的可靠性事故。

留意

  • 優化一項基準測試可以隱藏更廣泛的系統弱點。
  • 基礎設施和維護成本常常被低估。
  • 隨著系統變得更加複雜,安全性和可觀察性差距可能會擴大。

後續步驟

  1. 在實施之前定義延遲、品質和成本目標。
  2. 在實際負載和資料條件下進行基準測試。
  3. 儀器監控錯誤、漂移和使用者影響。
  4. 在擴展之前準備回滾和事件回應路徑。

不斷探索

免費電子報

獲取每日 AI 簡報

每個工作日上午提供三則經過驗證的 AI 新聞,以淺顯易懂的英文撰寫。永久免費,無廣告。

每個工作日一封電子郵件。一鍵取消訂閱。我們絕不出售或分享您的地址。

測試一下自己

Take the Retrieval Metrics: Recall@K, MRR and nDCG quiz

每個答案皆有即時回饋,通過課程後可獲得具備可驗證 ID 的可分享證書。

開始測驗

支持免費的 AI 教育。AI Understanding 是一個 501(c)(3) 非營利組織——絕無廣告,絕無付費牆。 捐款

常見問題

What is Retrieval Metrics: Recall@K, MRR and nDCG?

Recall@K measures how many known relevant items appear in the top K results, MRR emphasizes the rank of the first relevant item, and nDCG evaluates ranking quality using graded relevance with position discounts. Each metric reflects different judgments and use cases, and none directly measures whether a generated answer is correct.

What does Recall@K measure?

Recall at a cutoff measures relevant-item coverage in the top K.

What does Recall@K fail to distinguish by itself?

Recall@K measures coverage at the cutoff, not ordering within those results.

Why can incomplete relevance judgments distort retrieval metrics?

Metrics depend on the judged collection and can penalize unjudged positives.

Does a high retrieval Recall@K prove that a RAG answer is correct?

Retrieval and answer generation are distinct evaluation stages.