Technical GUIDE

Corrective RAG (CRAG)

3 min read
Last updated Sep 15, 2026

On this page3 min read

Overview
Deep Dive
Strategic Impact
The Future of Corrective RAG (CRAG)
Real-World Implementation
Risks & Guardrails
Implementation Roadmap
Keep Exploring
Frequently asked questions

Overview

It matters because standard RAG passes whatever it retrieves to the model, and irrelevant or misleading documents are a major cause of wrong, confidently stated answers.

Deep Dive

CRAG was introduced in the 2024 paper "Corrective Retrieval Augmented Generation" by Shi-Qi Yan and colleagues. It targets a basic weakness of standard RAG: the generator trusts the retriever. If the top documents are off-topic, outdated or only loosely related, the model often answers anyway. The paper's design adds a lightweight retrieval evaluator, a fine-tuned T5-large model in the original work, which scores each retrieved document for relevance to the question. Based on those scores, the system picks one of three actions. If at least one document is judged clearly relevant, the action is Correct: the documents go through knowledge refinement, a decompose-then-recompose step that splits them into small strips, scores each strip, and keeps only the useful ones. If all documents score low, the action is Incorrect: the retrieved documents are discarded, the question is rewritten into search-engine-style keywords, and web search results are used instead, again refined. If the evaluator is unsure, the action is Ambiguous, and the system combines refined internal knowledge with web results. The authors presented CRAG as plug-and-play: it can wrap standard RAG or other methods such as Self-RAG, and they reported improvements on several short-form and long-form generation benchmarks. In practice, many teams implement CRAG-style pipelines without the original fine-tuned evaluator, using an LLM prompt to grade relevance and a graph or agent framework (LangGraph has a well-known CRAG tutorial) to route between steps. Common misconceptions: CRAG does not verify that the final answer is correct; it grades retrieval inputs. Web search is not automatically better, since it can bring in low-quality sources. And grading adds latency and cost to every query.

Technical Insight

The central design choice is the grading threshold. A strict grader triggers fallback often, adding web calls and latency; a lax grader lets weak documents through, defeating the purpose. The original CRAG uses two thresholds on evaluator confidence to separate Correct, Ambiguous and Incorrect, and these need tuning for your data. LLM-based graders are easy to build, typically returning a binary or scored relevance judgment per chunk with structured output. They are slower and costlier than a small classifier or cross-encoder reranker, so high-volume systems often use a reranker score as the evaluator. Knowledge refinement can be approximated by sentence-level filtering, which also reduces tokens sent to the generator. For the fallback path, rewrite the question into concise keyword queries, restrict or rank domains you trust, and log which path each query took. That log shows whether your knowledge base has coverage gaps worth fixing directly.

Strategic Impact

Cost and budget

Architecture decisions drive performance and operating cost for years.

Clearer decisions

Technical education helps teams choose the right stack, not just the newest one.

Quality control

Better engineering choices reduce reliability incidents in production.

The Future of Corrective RAG (CRAG)

The CRAG idea, check retrieval quality and route accordingly, has become a common pattern in agentic RAG systems, where a model decides whether to retrieve again, search elsewhere or answer. Future work is likely to focus on cheaper and better-calibrated relevance evaluators, and on combining input checks with output checks such as verifying that the answer is supported by the kept evidence. The specific three-action scheme may matter less over time than the broader principle that retrieved context should be evaluated rather than trusted automatically.

Real-World Implementation

A company help-desk bot retrieves three policy chunks for a question about parental leave, the grader finds none relevant, and the system rewrites the query and searches again instead of answering from the wrong policy.

A medical-information assistant grades retrieved passages, keeps only the sentences that mention the drug the user asked about, and discards surrounding text about other drugs before generating.

A news Q&A tool detects that its internal archive has no coverage of a very recent event and falls back to a web search for current articles.

A legal research prototype marks retrieval as ambiguous when documents are only partly relevant, and combines refined internal passages with fresh search results before answering.

Risks & Guardrails

Optimizing one benchmark can hide broader system weaknesses.
Infrastructure and maintenance costs are often underestimated.
Security and observability gaps can grow as systems become more complex.

Implementation Roadmap

Define latency, quality, and cost targets before implementation.
Benchmark under realistic load and data conditions.
Instrument monitoring for errors, drift, and user impact.
Prepare rollback and incident response paths before scaling.

Keep Exploring

AI Benchmarks

Use evaluation properly when comparing technical options.Read Guide

Reinforcement Learning

Go deeper into technical training strategies.Read Guide

Free newsletter

One short email each weekday with the three AI stories that actually matter. Free forever, no ads.

One email each weekday. Unsubscribe in one click. We never sell or share your address.

Test yourself

Take the Corrective RAG (CRAG) quiz

Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.

Start quiz

Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation

Frequently asked questions

What is Corrective RAG (CRAG)?

Corrective RAG (CRAG) is a retrieval-augmented generation method that checks how relevant the retrieved documents are before answering, and takes a corrective action when retrieval looks weak, such as refining the documents, rewriting the query or searching the web. It matters because standard RAG passes whatever it retrieves to the model, and irrelevant or misleading documents are a major cause of wrong, confidently stated answers.

What core weakness of standard RAG does CRAG address?

Standard RAG passes retrieved documents to the model regardless of relevance, which can produce confident wrong answers. CRAG evaluates retrieval first.

In the original CRAG paper, what kind of model served as the retrieval evaluator?

The paper used a lightweight fine-tuned T5-large model to score document relevance.

What does CRAG do when all retrieved documents score low?

The Incorrect action discards the documents, rewrites the question into search queries and relies on refined web results.

What is the purpose of the decompose-then-recompose step?

Knowledge refinement breaks documents into small strips, scores them, and recomposes only the useful parts for the generator.

When the evaluator is unsure about relevance, which action does CRAG take?

The Ambiguous action blends both sources to hedge against uncertainty in the evaluator's judgment.

Keep learning