Understanding RAPTOR: Hierarchical Retrieval in AI

Recursive Abstractive Processing for Tree-Organized Retrieval (RAPTOR) is an advanced framework in artificial intelligence that improves how Retrieval-Augmented Generation (RAG) systems understand large documents. This article explains how RAPTOR overcomes the limitations of traditional retrieval methods by recursively clustering and summarizing text chunks into a hierarchical tree. By organizing information from granular details at the bottom to broad summaries at the top, RAPTOR allows AI models to efficiently answer both specific factual queries and complex, high-level thematic questions.

The Limitation of Traditional Retrieval

Standard RAG systems break documents into small, contiguous chunks of text and convert them into vector embeddings. When a user asks a question, the system fetches the chunks most semantically similar to the query. While this works well for simple fact-finding, it fails at answering high-level thematic questions. Thematic questions require synthesizing information spread across an entire document, and retrieving only a few isolated text chunks deprives the AI of the necessary overarching context.

Step 1: Chunking and Embedding the Base Layer

RAPTOR begins similarly to traditional systems. It takes a large document and splits it into small, manageable segments of text. These segments form the "leaf nodes" at the very bottom of the RAPTOR tree. Each leaf node is then converted into a vector embedding using an embedding model, capturing the semantic meaning of the raw text.

Step 2: Clustering Similar Information

Instead of leaving these chunks isolated, RAPTOR groups related chunks together, regardless of where they originally appeared in the document. It uses an algorithm called Gaussian Mixture Models (GMM) combined with UMAP for dimensionality reduction.

A critical feature of RAPTOR's clustering is the use of "soft clustering." Because a single paragraph might touch on multiple themes, soft clustering allows a single text chunk to be assigned to multiple clusters simultaneously. This ensures that vital bridging concepts are not restricted to just one thematic pathway.

Step 3: Summarizing the Clusters

Once the chunks are grouped into clusters, RAPTOR uses a Large Language Model (LLM) to generate a concise summary for each cluster. This process distills the granular details of the individual chunks into a broader, more abstract representation of their shared theme. These generated summaries become the new nodes in the next layer up the tree.

Step 4: Building the Tree Recursively

RAPTOR repeats the embedding, clustering, and summarizing process on the newly created summaries.

  1. The new summaries are converted into vector embeddings.
  2. They are grouped into new, broader clusters based on semantic similarity.
  3. The LLM summarizes these new clusters to form an even higher layer.

This recursive cycle continues until the remaining nodes can no longer be clustered meaningfully, culminating in a single root node or a small group of top-level nodes that summarize the entire document. The result is a multi-layered hierarchical tree: the bottom leaves hold exact details, the middle layers hold sub-themes, and the top layers hold overarching themes.

Querying the Tree for Thematic Answers

When a user asks a question, RAPTOR can retrieve information from the tree using two primary methods: tree traversal and collapsed tree retrieval.

In tree traversal, the system starts at the top layer of the tree, selects the nodes most relevant to the query, and drills down into their child nodes to gather the best mix of summaries and raw text.

In collapsed tree retrieval, the system flattens the entire tree into a single pool of nodes. It then compares the user's query against every node—both original chunks and all levels of summaries—simultaneously.

Because high-level summaries exist alongside specific text chunks, a thematic question will naturally match with the higher-level summary nodes. The AI receives pre-synthesized, overarching context rather than fragmented details, allowing it to generate accurate, comprehensive answers to complex thematic questions.