How Corrective RAG Filters Irrelevant AI Data

Corrective Retrieval-Augmented Generation (CRAG) enhances artificial intelligence accuracy by introducing a lightweight retrieval evaluator to assess fetched documents before generating a final response. While standard Retrieval-Augmented Generation (RAG) blindly feeds retrieved passages into a Large Language Model (LLM), CRAG actively grades the relevance of this data first. Based on the evaluator's confidence score, the system filters out irrelevant noise, isolates useful facts, or triggers an external web search to find better information, ensuring the AI synthesizes answers exclusively from verified and accurate context.

Standard RAG systems frequently suffer from retrieval failures. When a user asks a question, the system pulls documents from a vector database based on semantic similarity. However, similarity does not always equal relevance. If the retrieved passages are inaccurate or off-topic, the LLM is forced to synthesize an answer using flawed data, which leads to hallucinations. CRAG solves this by placing a gatekeeper between the retrieval step and the generation step.

This gatekeeper is a lightweight retrieval evaluator. Unlike massive LLMs that require significant computing power, the evaluator is typically a smaller, fine-tuned model—such as a T5 architecture—designed for one specific task: classifying the relevance of a document in relation to a user's query. Because the model is small, it processes retrieved passages quickly and cheaply without adding noticeable latency to the pipeline.

When passages are retrieved, the lightweight evaluator analyzes the query and the document together, outputting a numerical confidence score. Based on predefined thresholds, the evaluator assigns one of three action triggers: Correct, Incorrect, or Ambiguous.

If the evaluator scores the retrieved passages above the upper threshold, it triggers the "Correct" action. Even when a document is deemed relevant, it often contains unnecessary fluff alongside the answer. To maximize precision, CRAG applies a decompose-then-recompose algorithm. It breaks the long passage down into smaller segments, runs those segments through the evaluator again to strip away irrelevant sentences, and recomposes only the most critical facts. This refined text is then passed to the LLM.

If the evaluator scores all retrieved passages below the lower threshold, it triggers the "Incorrect" action. The system immediately discards the retrieved passages, preventing the LLM from ever seeing the bad data. Instead of guessing, CRAG rewrites the user's prompt into a keyword-optimized query and uses an external web search API to fetch accurate information from the internet.

If the confidence score falls between the upper and lower thresholds, the evaluator triggers the "Ambiguous" action. This happens when the retrieved data is only partially relevant or lacks sufficient detail to fully answer the prompt. In this scenario, CRAG combines the other two approaches. It uses the decompose-then-recompose method to extract whatever useful information exists in the retrieved passages, while simultaneously executing a web search to fill in the missing knowledge gaps.

Once the lightweight evaluator has routed the data and executed the necessary filtering or searching, the resulting information is compiled. The LLM is finally given the original prompt alongside this heavily curated context. By actively grading and purifying the retrieved passages beforehand, CRAG guarantees that the final AI synthesis is grounded in high-quality, relevant facts.