OkulandelayoUmhlahlandlela olandelayo
RAG Evaluation Metrics and RAGAS
Ubuchwepheshe
UMHLAHLANDLELA Wobuchwepheshe
I-RAG Yokulungisa (CRAG)
Corrective RAG (CRAG) is a retrieval-augmented generation method that checks how relevant the retrieved documents are before answering, and takes a corrective action when retrieval looks weak, such as refining the documents, rewriting the query or searching the web.
Kuleli khasi3 min ifundiwe
Uhlolojikelele
It matters because standard RAG passes whatever it retrieves to the model, and irrelevant or misleading documents are a major cause of wrong, confidently stated answers.
I-Deep Dive
CRAG was introduced in the 2024 paper "Corrective Retrieval Augmented Generation" by Shi-Qi Yan and colleagues. It targets a basic weakness of standard RAG: the generator trusts the retriever. If the top documents are off-topic, outdated or only loosely related, the model often answers anyway. The paper's design adds a lightweight retrieval evaluator, a fine-tuned T5-large model in the original work, which scores each retrieved document for relevance to the question. Based on those scores, the system picks one of three actions. If at least one document is judged clearly relevant, the action is Correct: the documents go through knowledge refinement, a decompose-then-recompose step that splits them into small strips, scores each strip, and keeps only the useful ones. If all documents score low, the action is Incorrect: the retrieved documents are discarded, the question is rewritten into search-engine-style keywords, and web search results are used instead, again refined. If the evaluator is unsure, the action is Ambiguous, and the system combines refined internal knowledge with web results. The authors presented CRAG as plug-and-play: it can wrap standard RAG or other methods such as Self-RAG, and they reported improvements on several short-form and long-form generation benchmarks. In practice, many teams implement CRAG-style pipelines without the original fine-tuned evaluator, using an LLM prompt to grade relevance and a graph or agent framework (LangGraph has a well-known CRAG tutorial) to route between steps. Common misconceptions: CRAG does not verify that the final answer is correct; it grades retrieval inputs. Web search is not automatically better, since it can bring in low-quality sources. And grading adds latency and cost to every query.
I-Strategic Impact
Izindleko kanye nesabelomali
Izinqumo zezakhiwo ziqhuba ukusebenza kanye nezindleko zokusebenza iminyaka.
Izinqumo ezicacile
Imfundo yobuchwepheshe isiza amaqembu ukuthi akhethe isitaki esifanele, hhayi nje esisha.
Ukulawulwa kwekhwalithi
Izinketho ezingcono zobunjiniyela zinciphisa izehlakalo ezinokwethenjelwa ekukhiqizeni.
The Future of Corrective RAG (CRAG)
The CRAG idea, check retrieval quality and route accordingly, has become a common pattern in agentic RAG systems, where a model decides whether to retrieve again, search elsewhere or answer. Future work is likely to focus on cheaper and better-calibrated relevance evaluators, and on combining input checks with output checks such as verifying that the answer is supported by the kept evidence. The specific three-action scheme may matter less over time than the broader principle that retrieved context should be evaluated rather than trusted automatically.
Ukuqaliswa Komhlaba Wangempela
A company help-desk bot retrieves three policy chunks for a question about parental leave, the grader finds none relevant, and the system rewrites the query and searches again instead of answering from the wrong policy.
A medical-information assistant grades retrieved passages, keeps only the sentences that mention the drug the user asked about, and discards surrounding text about other drugs before generating.
A news Q&A tool detects that its internal archive has no coverage of a very recent event and falls back to a web search for current articles.
A legal research prototype marks retrieval as ambiguous when documents are only partly relevant, and combines refined internal passages with fresh search results before answering.
Izingozi & Guardrails
Ukuthuthukisa ibhentshimakhi eyodwa kungafihla ubuthakathaka obubanzi besistimu.
Izindleko zengqalasizinda nezokulungisa zivame ukubukelwa phansi.
Izikhala zokuphepha nokubonakala zingakhula njengoba izinhlelo ziba nzima kakhulu.
Ukuqalisa Umhlahlandlela
Chaza ukubambezeleka, ikhwalithi, nezindleko ezihlosiwe ngaphambi kokuqaliswa.
Ibhentshimakhi ngaphansi komthwalo wangempela nezimo zedatha.
Ukuqapha amathuluzi amaphutha, ukukhukhuleka, nomthelela wabasebenzisi.
Lungiselela izindlela zokuhlehlisa nezigameko ngaphambi kokukala.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Corrective RAG (CRAG) quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Imibuzo evame ukubuzwa
What is Corrective RAG (CRAG)?
Corrective RAG (CRAG) is a retrieval-augmented generation method that checks how relevant the retrieved documents are before answering, and takes a corrective action when retrieval looks weak, such as refining the documents, rewriting the query or searching the web. It matters because standard RAG passes whatever it retrieves to the model, and irrelevant or misleading documents are a major cause of wrong, confidently stated answers.
What core weakness of standard RAG does CRAG address?
Standard RAG passes retrieved documents to the model regardless of relevance, which can produce confident wrong answers. CRAG evaluates retrieval first.
In the original CRAG paper, what kind of model served as the retrieval evaluator?
The paper used a lightweight fine-tuned T5-large model to score document relevance.
What does CRAG do when all retrieved documents score low?
The Incorrect action discards the documents, rewrites the question into search queries and relies on refined web results.
What is the purpose of the decompose-then-recompose step?
Knowledge refinement breaks documents into small strips, scores them, and recomposes only the useful parts for the generator.
When the evaluator is unsure about relevance, which action does CRAG take?
The Ambiguous action blends both sources to hedge against uncertainty in the evaluator's judgment.
Qhubeka ufunda
Iziqondiso ezihlobene
Imihlahlandlela eyengeziwe yalesi sihloko