PageIndex: The Vectorless, Reasoning-Based RAG Alternative (2026 Guide)

For three years, "do RAG" has meant one recipe: chop your documents into chunks, turn each chunk into a vector with an embedding model, stuff those vectors into a database, and at question time retrieve the chunks whose vectors sit closest to the question's vector. It works. It also loses to a surprisingly simple idea.

PageIndex, an open-source project from VectifyAI, deletes the entire vector stack. No embeddings, no chunking, no Pinecone or Chroma. Instead it turns each document into a hierarchical tree — essentially a machine-readable table of contents — and lets an LLM reason its way down the branches to the exact section that answers your question, the way a human analyst flips to the right chapter. On the FinanceBench document-QA benchmark, a system built on this approach reported 98.7% accuracy, against roughly 50% for typical vector RAG. This guide explains how it works, why "similarity is not relevance" is the whole ballgame, when to reach for it, and how to run it yourself — including against a local model.

The problem PageIndex is trying to solve

To see why a vectorless approach is interesting, you have to be honest about how ordinary RAG behaves under pressure.

Classic Retrieval-Augmented Generation is a four-step pipeline. You split documents into fixed-size chunks (say, 500 tokens). You run each chunk through an embedding model to get a vector — a list of numbers that encodes roughly what the text is "about." You store those vectors in a vector database. At query time you embed the question the same way and retrieve the top-k chunks whose vectors are nearest by cosine similarity. Those chunks become the model's context. If you want the full mechanics, our Complete Local RAG Pipeline walks through every stage.

This pipeline has three failure modes that get worse exactly when the documents get valuable:

• Chunking destroys structure. A 500-token window has no idea it just sliced a financial table in half, orphaned a clause from its heading, or split "Risk Factors" across three unrelated chunks. The model receives fragments, not sections.
• Similarity is not relevance. Embeddings reward surface-level semantic overlap. Ask "What was the company's free cash flow in FY2024?" and the retriever may hand you five paragraphs that talk about cash flow while missing the one line in the cash-flow statement that contains the number. The vectors were "close." They were also wrong.
• It is a black box. When retrieval fails, you get a confidently wrong answer with no audit trail. You cannot ask the vector store why it picked those chunks. For a regulated report, "the cosine distance was 0.83" is not an explanation anyone will accept.

None of this means embeddings are bad. It means embeddings answer the question "which text looks similar?" when what you often need is "which section of this document should an expert actually read?" Those are different questions. PageIndex is a bet that the second one is better answered by reasoning than by geometry.

What PageIndex actually is

PageIndex is an open-source framework for vectorless, reasoning-based RAG. The name is the whole idea: instead of an index of vectors, it builds an index of pages — a hierarchical tree that mirrors the document's own structure, like a detailed table of contents where every entry knows its title, its page range, and a short summary of what lives inside.

Retrieval then becomes tree search performed by an LLM. Given the tree, the model reasons about which branches are worth opening — "the question is about liquidity, so the Financial Statements → Cash Flow section is the place to look" — walks down to the relevant nodes, and pulls their text to ground the final answer. There is no embedding step anywhere in the loop.

That is the entire architecture. It is deliberately close to how a person works: you do not embed a 10-K into a coordinate space in your head — you open the table of contents, decide where the answer probably lives, and read that part.

How PageIndex works, step by step

Step 1 — Build the tree index

The first stage runs once per document. PageIndex parses the file, detects its structure (using the real table of contents when there is one, and inferring section boundaries when there is not), and emits a tree of nodes. Each node describes one section: its title, the page range it spans, a short LLM-written summary of what the section says, and an ordered list of child nodes for its subsections.

Here is a single node from a real tree, taken from the project's own example over a Federal Reserve report:

{
  "title": "Financial Stability",
  "node_id": "0006",
  "start_index": 21,
  "end_index": 22,
  "summary": "The Federal Reserve monitors vulnerabilities in the financial system and works to ensure it can withstand shocks...",
  "nodes": [
    {
      "title": "Monitoring Financial Vulnerabilities",
      "node_id": "0007",
      "start_index": 22,
      "end_index": 28,
      "summary": "The Federal Reserve's monitoring framework tracks asset valuations, borrowing by businesses and households..."
    }
  ]
}

Read those fields and you can already see why the design works. start_index and end_index are literal page numbers, so every answer can cite an exact page. summary is what the LLM reads during search — it is the "blurb" that lets the model judge relevance without loading the section's full text. And nodes gives the hierarchy, so a question about a sub-topic can be routed down to the right subsection instead of dumping a whole chapter into context.

You control how the tree is shaped. The indexer exposes options for how many pages it scans looking for a table of contents, the maximum pages and tokens allowed per node (which caps how big any single retrieved section can get), and whether to attach node IDs, summaries, and a document-level description. We list the exact flags in the hands-on section below.

Step 2 — Reasoning-based retrieval (tree search)

The second stage runs on every question. PageIndex hands the LLM the tree — the titles and summaries, not the full document — and asks it to do what a person does with a table of contents: figure out where the answer lives.

Concretely, the loop is:

1. Present the tree. The model sees the hierarchy of node titles and summaries (compact enough to fit in context even for a long document, because summaries are short). 2. Reason about relevance. The model thinks through the question against the structure and decides which node or nodes are worth opening. This is real reasoning, not a distance calculation — it can follow multi-hop logic ("net leverage means I need both total debt from the Balance Sheet and EBITDA from the Income Statement"). 3. Return node IDs. The model outputs the node_id values it wants, so the selection is explicit and inspectable. 4. Fetch and answer. PageIndex pulls the full text of exactly those nodes (using their page ranges) and passes it to the model to write the final, grounded answer — accompanied by the node references it walked, which is your audit trail.

Because the selection is a list of node IDs tied to page numbers, you can always answer the question every compliance officer eventually asks: where did this come from? The answer is "nodes 0006 and 0031, pages 21–22 and 87–90" — not a similarity score.

Why "similarity ≠ relevance" is the whole point

This phrase is doing a lot of work, so it is worth slowing down on. An embedding model maps text to a point in a few-thousand-dimensional space such that texts about similar things land near each other. That is genuinely useful — it is why vector search feels like magic on open-domain questions. But "about a similar thing" and "contains the specific answer" diverge hard on professional documents:

• The defining sentence is often terse. The number you need might be a single cell in a table whose surrounding text is sparse. A chatty paragraph elsewhere that merely discusses the topic will out-score it on cosine similarity.
• Reasoning is multi-hop. "Is the covenant breached?" requires finding a threshold in the credit agreement and a figure in the financials and comparing them. No single chunk is "similar" to that question; the answer is assembled by reasoning across sections — exactly what tree search does and vector top-k does not.
• Structure carries meaning. In a contract, which section a clause sits in changes what it means. Embeddings flatten that away; a tree preserves it.

PageIndex vs traditional vector RAG

Here is the head-to-head. Neither column is universally "better" — they optimize for different shapes of problem, which is the real takeaway.

The pattern is clear: vector RAG is a fast, cheap, scalable filter that works best when you have an enormous pile of loosely related snippets and semantic similarity is a good-enough proxy for relevance. PageIndex is a deep, accurate reader that works best when the document is long, structured, and the cost of a wrong answer is high. Notice that "cost per query" and "scaling" are the columns where vector RAG wins — that is not a footnote, it is the reason both approaches will keep existing.

The benchmark that got everyone's attention: FinanceBench

The result that put PageIndex on the map: Mafin 2.5, a financial document-analysis system VectifyAI built on top of PageIndex, reported 98.7% accuracy on FinanceBench, a benchmark of real questions over SEC filings and financial reports. For context, the same source reports traditional vector-based RAG landing around 50% on the same benchmark, and a strong model answering closed-book (no retrieval) around 31%.

Two things are true at once here, and a good engineer holds both. First, the gap is large enough that it is not noise — reasoning over structure clearly helps a lot on this class of documents. Second, these are vendor-reported numbers on a single domain (financial filings, which happen to be beautifully structured), and Mafin 2.5 is a full production system, not the bare open-source library. So read 98.7% as "this approach can be dramatically better where structure is rich," not as "PageIndex scores 98.7% on your data out of the box." Your mileage depends on how well your documents carry structure. For more on how benchmark claims should be read skeptically, see DeepSeek R1 vs GPT-4o.

Hands-on: build your first tree index

PageIndex is a normal Python project. The indexing step is a single command.

# 1. Install
git clone https://github.com/VectifyAI/PageIndex.git
cd PageIndex
pip3 install --upgrade -r requirements.txt

# 2. Configure an LLM (multi-provider via LiteLLM)
echo "OPENAI_API_KEY=sk-your-key-here" > .env

# 3. Build a tree from a PDF
python3 run_pageindex.py --pdf_path /path/to/annual-report.pdf

That writes a JSON tree like the node example above. The generator is configurable; these are the flags you will actually touch:

Working with Markdown or HTML instead of PDF? Point it at --md_path and skip PDF parsing entirely — handy when your source is already clean text, which also sidesteps the OCR quality issues that plague PDF pipelines. The node summaries are the highest-leverage setting: they are all the model reads during search, so a good model at index time pays off on every future query.

Reasoning-based retrieval in code

Once you have a tree, retrieval is "give the LLM the tree, get back node IDs, fetch those nodes, answer." Here is a minimal, illustrative version of that loop so you can see there is genuinely no vector math involved:

import json
from openai import OpenAI

client = OpenAI()  # or point base_url at any OpenAI-compatible endpoint
tree = json.load(open("annual-report.tree.json"))

def outline(nodes, depth=0):
    """Flatten the tree into an id/title/summary outline for the model to read."""
    lines = []
    for n in nodes:
        pad = "  " * depth
        lines.append(f"{pad}[{n['node_id']}] {n['title']} -- {n['summary']}")
        lines += outline(n.get("nodes", []), depth + 1)
    return "\n".join(lines)

def ask(question):
    # Step 1: the model reasons over the outline and returns relevant node IDs.
    search = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content":
            "Document outline (id, title, summary):\n" + outline(tree) +
            f"\n\nQuestion: {question}\n"
            "Reply with ONLY a JSON list of the node_ids needed to answer."}],
    )
    node_ids = json.loads(search.choices[0].message.content)

    # Step 2: fetch the full text of exactly those nodes (by their page ranges),
    # then answer, grounded and cited. get_text() is your PDF/page extractor.
    context = "\n\n".join(get_text(nid) for nid in node_ids)
    answer = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content":
            f"Using only this context:\n{context}\n\nAnswer: {question} "
            "Cite the node_ids you used."}],
    )
    return answer.choices[0].message.content, node_ids

The real project ships more robust versions of this — including an agentic example built on the OpenAI Agents SDK where the model can iteratively open nodes, realize it needs a sibling section, and keep traversing until it has enough. That iterative traversal is where multi-hop questions get answered, and it is the part vector top-k fundamentally cannot do. If you build agentic retrieval, our Multi-Agent LLM Systems guide covers the orchestration patterns.

Running PageIndex against a local model

Here is the part that makes PageIndex especially interesting for anyone who cares about privacy or cost: vectorless means there is nothing to self-host but the model. Classic local RAG asks you to run an embedding model and stand up a vector database (Chroma, pgvector) and keep them in sync. PageIndex deletes both. You need one thing — a capable reasoning LLM — and it can be a local one.

Because PageIndex talks to models through an OpenAI-compatible interface, you can point it at a local server. If you run Ollama, it exposes exactly that endpoint:

# Serve a strong reasoning model locally
ollama pull qwen3
# Ollama's OpenAI-compatible API lives at http://localhost:11434/v1

Then set your client's base_url to http://localhost:11434/v1 and a placeholder API key, and both the tree-building and the tree-search calls run on your own hardware — no document, question, or answer ever leaves the machine. That is a genuinely private document-analysis pipeline with far fewer moving parts than a vector stack.

One honest caveat: tree search leans hard on reasoning quality, so a small local model will build worse trees and make worse node selections than a frontier model. This approach rewards a capable model more than vector RAG does. If your local model is modest, consider building the tree with a stronger model once (indexing is a one-time cost) and doing per-query search locally.

When PageIndex beats RAG — and when it doesn't

The "PageIndex kills RAG" headlines oversell it. The accurate framing is that it is excellent in specific cases and unremarkable in others. Match the tool to the shape of your problem.

A useful mental model: PageIndex is not a faster horse, it is a different animal. Vector RAG is a librarian who is very fast at pointing you to books that mention your topic. PageIndex is an analyst who slowly reads the right book and reasons about it. You want the librarian for "find me anything about X across a million documents" and the analyst for "answer this precise question about this specific 200-page document, and show your work."

The cost and latency reality check

Do not skip this section. The single biggest tradeoff PageIndex makes is that retrieval is now LLM inference, not a database lookup.

• Latency. A vector search returns in milliseconds. Tree search runs one or more LLM calls per query, sometimes sequentially as the model traverses — that is seconds, not milliseconds. For interactive chat over one document it is fine; for a high-QPS search box it is not.
• Cost. Every query spends tokens (reading the outline, reasoning, then reading the selected sections and answering). Vector RAG spends a fraction of a cent on the lookup and only pays the LLM once, at answer time. If you serve millions of queries, this difference dominates.
• Indexing cost. Building the tree is itself an LLM job. It is a one-time cost per document, but it is real, and it scales with document length.
• No public latency benchmarks. VectifyAI has published accuracy numbers but not, as of this writing, throughput or latency figures. Budget for "noticeably slower than vector search" and measure on your own documents.

The mitigations are sensible: build trees offline and cache them, use a cheaper model for search and a stronger one only for the final answer, and keep documents scoped so outlines stay small. But the core tradeoff is inherent — you are buying accuracy and explainability with compute.

You don't have to choose: hybrid retrieval

The framing of "PageIndex replaces RAG" is a useful provocation, but the mature architecture is often both. A very effective pattern at scale:

1. Vector search as the coarse filter. Over a large corpus, use fast embedding search to narrow millions of documents down to the handful that are probably relevant. This is what vectors are genuinely great at. 2. PageIndex as the deep reader. Within those few documents, use tree search to reason to the exact sections and produce a grounded, cited answer. This is what reasoning is genuinely great at.

That gives you vector search's scalability and PageIndex's precision and explainability, each doing the job it is best at. If you already run a local RAG pipeline, you can bolt PageIndex on as the second stage without ripping anything out — the vector store keeps earning its keep as a document-level filter while PageIndex handles the within-document reasoning it does better.

Deployment options

PageIndex meets you at three levels:

• Self-hosted (open source). The GitHub repo is the full pipeline — clone it, run run_pageindex.py, own everything. Best for privacy, local models, and tinkering.
• Cloud service. VectifyAI runs a hosted pipeline with stronger OCR and tree-building, reachable via a REST API and a chat interface. Best when your PDFs are messy and you would rather not babysit parsing.
• MCP and agents. PageIndex exposes a Model Context Protocol server, so agents can call tree search as a tool, and ships examples using the OpenAI Agents SDK for autonomous, multi-step document reasoning.

For a project that is a competing philosophy to the entire vector-database industry, it is refreshingly easy to try: one pip install, one command, and you have a tree you can inspect by hand.

Frequently asked questions

Is PageIndex really "vectorless"?

Yes. There are no embeddings and no vector database anywhere in the pipeline. The only index is a JSON tree of titles, page ranges, and summaries, and retrieval is LLM reasoning over that tree. The tradeoff is that "retrieval" now costs LLM calls instead of a vector lookup.

Does PageIndex replace RAG entirely?

It replaces the vector-similarity machinery of RAG — chunking, embeddings, and the vector store — with reasoning over structure. It is still retrieval-augmented generation in the broad sense (you retrieve relevant context and generate an answer). For huge, unstructured corpora where speed and scale matter most, classic vector RAG is still the right tool, and hybrids are common.

What documents work best?

Long, structured, professional documents: SEC filings and annual reports, legal contracts, regulatory filings, technical and equipment manuals, medical literature, and textbooks. The richer the inherent structure, the better the tree — and the bigger the advantage over chunk-and-embed.

How does it stay explainable?

Every answer is produced from a specific set of node IDs, each tied to a page range. You can see exactly which sections the model read and cite them — a real audit trail, which is why the approach is attractive in finance, law, and healthcare.

Can I run it with a local, open-source model?

Yes — that is one of its best properties. Because there is no embedding model or vector DB to host, you only need one reasoning LLM, and PageIndex speaks the OpenAI-compatible API, so you can point it at Ollama or vLLM. Use a capable model: tree quality and node selection depend heavily on reasoning ability.

What is the catch?

Latency and cost. Retrieval is LLM inference, so it is seconds-not-milliseconds and spends tokens on every query, plus a one-time LLM cost to build each tree. If you need millisecond retrieval at high volume, that is the wrong trade.

Key takeaways

• PageIndex is vectorless, reasoning-based RAG: build a table-of-contents tree per document, then let an LLM reason its way to the right section instead of matching embedding vectors.
• "Similarity is not relevance" is the core insight — on structured documents, the section a human would open beats the chunk that looks most similar.
• The upside is accuracy, explainability, and simplicity: no vector DB, no embedding model, no chunking, and a traceable page-level audit trail. A reported 98.7% on FinanceBench shows how large the gap can be where structure is rich.
• The downside is cost and latency: retrieval is LLM inference, so budget for seconds and tokens per query, and a one-time indexing cost.
• It pairs beautifully with local models, because there is nothing to self-host but the LLM itself.
• You often want both: vector search to filter a big corpus, PageIndex to deep-read the survivors.

Where to go next: stand up the model layer with the Complete Ollama Bible, compare it against a classic pipeline in the Complete Local RAG Pipeline, and if you are serving many users at once, weigh vLLM vs Ollama for the inference backend.

Sources & further reading

• VectifyAI/PageIndex on GitHub — the open-source repository, node structure, and CLI.
• VectifyAI launches Mafin 2.5 and PageIndex — 98.7% on FinanceBench (MarkTechPost) — the benchmark announcement.
• FinanceBench (patronus-ai) — the financial-document QA benchmark referenced above.
• "No, PageIndex will not kill RAG, but it is indeed excellent in some cases" (Alden Do Rosario) — a balanced critique worth reading.