Retrieval-Augmented Generation (RAG)

It can give the system access to relevant documents without retraining the model for every document change. Retrieval does not guarantee that the final answer is supported or correct.

Key takeaways

• Keep provenance and document context.
• Enforce permissions before retrieval results reach the model.
• Evaluate retrieval and generation independently.

A typical pipeline collects documents, preserves their provenance, creates searchable representations, retrieves candidates for a query, and passes selected evidence to a model. Some systems combine keyword and semantic search or rerank results before generation. Each stage can introduce omissions or errors. Document preparation determines what evidence can be found. Preserve headings, dates, tables, and source identifiers when dividing content into passages. A passage separated from an exception or footnote can convey the wrong meaning even when the words are copied correctly. Apply access controls before evidence reaches the model. A search result that is semantically relevant may still belong to a document the requesting user is not allowed to see. Treat instructions inside retrieved documents as untrusted content rather than authority to change the application’s behavior. Evaluate retrieval and answering separately. Check whether the needed evidence appears in the candidate set, whether the selected context retains it, and whether the answer uses it faithfully. Include questions with no answer in the collection and conflicting or outdated documents. The system needs an explicit way to say that evidence is insufficient.

1. Construct two policies: an expired version allows returns for 30 days; the current version allows 14 days.
2. If retrieval returns only the old policy, the failure is upstream of generation. If both are retrieved but the answer chooses 30 days, investigate context selection and evidence use.
3. Add the effective-date conflict to both retrieval and answer evaluations.

The invented example separates two causes that would otherwise look like the same wrong answer.