How to Select a Retrieval Strategy for Your AI Agents

A practical guide from the Blackburn Labs team. We connect AI agents to company data on many projects. This guide shows the four patterns we use, the benefits and limits of each, and a simple method to select between them.

Here is a common scene. A team builds an AI assistant for their company. The demo is impressive. The assistant answers questions about company documents with speed and confidence. Then the assistant goes into production. Users ask real questions. The assistant gives answers that sound correct but are not correct. It cannot find documents that everyone knows exist. Trust decreases. The project stops.

The team did not fail because AI is not ready. The team failed because they selected the wrong retrieval strategy and did not measure retrieval quality. This guide will help you prevent both problems.

In this guide:

1. Retrieval is a tool, not an architecture
2. Lane 1: Long Context
3. Lane 2: Hybrid RAG
4. Lane 3: Graph-Enhanced Retrieval
5. Lane 4: Agentic Retrieval
6. The most important step: measure
7. How to select your lane

Retrieval Is a Tool, Not an Architecture

Two years ago, teams spoke about RAG (Retrieval-Augmented Generation) as the architecture of an AI system. You built “a RAG pipeline,” and the pipeline was the product.

The industry view has changed, and we agree with the change. Today, retrieval is one tool that your AI agent uses. The important discipline is context engineering: controlling everything the model sees. This includes instructions, available tools, memory, and retrieved data. Retrieval is one input among several.

This change is not only a change of words. It changes how you make decisions:

• You do not select one pipeline forever. You select the simplest retrieval method that answers your questions today. You can change the method later without a full rebuild.
• Your agent can use more than one method. A modern agent can search an index for one question and read a full document for the next question.
• The quality of the answer depends on the full context. A perfect search result does not help if the instructions are unclear or the context contains unnecessary data.

We organize retrieval methods into four lanes. The lanes go from simple to complex. Our strong recommendation: start in the simplest lane that answers your questions. Move to a more complex lane only when your questions make it necessary.

Lane 1: Long Context

In short: no pipeline. Load the documents.

What it is

Modern language models accept very large inputs. Many models accept hundreds of pages of text in one request. If your document set is small, you do not need a retrieval pipeline at all. You put all of the documents into the model context, and the model reads everything.

This option did not exist when RAG became popular. Many teams still build pipelines that they do not need. Do not be one of those teams.

When to use it

Use Lane 1 when your corpus is small and known. Hundreds of pages is a good size. Thousands of pages is possible with some models. Millions of pages is not possible. All of the answers must be in one known set of documents.

How it operates

The flow is short. The user asks a question. The system puts the full document set into the context. A prompt cache keeps the cost of repeat calls low. The model reads all of the documents once and gives an answer based on the full corpus.

Benefits and limits

• Benefit: There is no infrastructure. There is nothing to build. There are no document chunks, no embeddings, and no index that can become outdated. The documents in the context are always the documents that you loaded.
• Limit: Each call uses many tokens, so each call has a real cost. The corpus size has a firm limit. When your document set grows past the context window, this lane stops.

Applications

Policy questions and answers. The documents of a single deal or transaction. Employee handbooks. Any bounded set of reference material.

Lane 2: Hybrid RAG

In short: retrieval with correct engineering. This is our default.

What it is

This is the classic search-then-answer pattern, but with the engineering quality that production systems need in 2026. The word “hybrid” is important. A simple vector search alone is not sufficient. Production systems combine vector search (which finds meaning) with keyword search (which finds exact terms, nouns, names, codes, etc). A reranker then scores the combined candidates and sorts them.

We see many systems that use only vector search with a “top-k” cutoff. That was acceptable in 2023. It is below the accepted quality level now.

When to use it

Use Lane 2 when you have a large corpus and mostly single-hop questions. A single-hop question is a question that one passage can answer. Example: “What is the termination clause in the standard vendor agreement?” The system must find the correct passage; the model then writes the answer from that passage.

How it operates

The user asks a question. The system runs a hybrid search: vector search plus keyword search (BM25 is the common algorithm). A reranker scores and sorts the candidate passages. The model receives only the top passages and writes an answer with references to the sources.

Benefits and limits

• Benefit: This lane gives the best balance of cost and quality at scale. It answers questions across millions of pages for a few cents per query.
• Limit: Two areas need continuous attention. First, chunk quality: the method that divides documents into pieces has a large effect on results. Second, index updates: when documents change, the index must change with them. Also, this lane answers single-hop questions only. It cannot connect facts across many documents.

Applications

Support assistants. Document search across large repositories. Policy and procedure lookups.

Lane 3: Graph-Enhanced Retrieval

In short: use when relations are important. Use only if necessary.

What it is

Some questions are not about one passage. They are about connections. “Which vendors are connected to the entities in these three agreements?” A vector search cannot answer this question, because the answer is not in one place. It is spread across entities and their relations.

Graph-enhanced retrieval extracts entities (people, companies, documents, dates) and their relations into a knowledge graph. The graph then guides retrieval across multiple hops.

A caution before you start

This is the lane where we most often see wasted money. Full graph construction with automatic entity extraction, community detection, and hierarchical summaries is expensive. Some published methods cost 10 to 40 times more than vector indexing for the same corpus, and the graph then needs continuous maintenance. Newer, lighter methods give most of the benefit at a small part of the cost.

Our advice is direct: start with a small graph. Add only the entities and relations that your real questions need. Do not build the full graph because a diagram on the internet showed one.

How it operates

The user asks a question. The system extracts entities and puts them into a small knowledge graph. The graph guides multi-hop retrieval: it follows relations from entity to entity and collects the connected context. The model receives this connected context and writes an answer that covers entities and their relations.

Benefits and limits

• Benefit: This lane makes multi-hop, cross-entity questions possible. Vector search cannot do this.
• Limit: Graphs are expensive to build and maintain. The schema needs design. The extraction needs review. The maintenance continues for the life of the system.

Applications

Legal entity analysis. Fraud pattern detection. Supply chain questions. Any domain where the relations between records hold the answers.

Lane 4: Agentic Retrieval

In short: retrieval in a reasoning loop. The most capable lane.

What it is

In the first three lanes, retrieval is one fixed step. In Lane 4, a reasoning agent controls the retrieval. The agent examines the task, selects a method, runs a search, examines the results, and then decides what to do next. It can search again with better terms. It can use a different tool. It can connect to live systems through MCP (Model Context Protocol) and read current data, not only an index from yesterday.

This is the lane that people mean when they speak about “AI agents” today. It also receives the most attention in the market, so the limits deserve equal examination.

When to use it

Use Lane 4 for complex tasks across many separate sources, when quality is more important than speed or cost. Example: “Review this agreement against our past agreements with this vendor, and identify the terms that changed.” No single search answers this task. The work needs a sequence of searches, filters, comparisons, and checks.

How it operates

The user gives a complex task. A reasoning agent selects the method. The agent uses its tools: index search, graph queries, and live systems through MCP. The agent evaluates its own results, searches again when necessary, and repeats until the task is complete. The answer receives a check before delivery.

Benefits and limits

• Benefit: This lane has the highest quality ceiling. It connects to live systems. It recovers from bad first searches. It completes tasks that fixed pipelines cannot complete.
• Limit: The costs are real. An agentic task uses 3 to 10 times more tokens than a single retrieval call. It is slower; tasks take seconds to minutes, not milliseconds. The results vary between runs, because the agent makes choices. For all of these reasons, evaluation is necessary. Do not ship an agentic system without an evaluation harness.

Applications

Research workflows. Contract analysis. Tasks that span multiple systems.

The Most Important Step: Measure

We will state our strongest opinion in plain words. Select a lane. Then measure it.

Retrieval evaluation and monitoring are more important than the architecture choice. A team with a simple lane and good measurement will get better results than a team with a complex lane and no measurement. If you do not measure retrieval quality, you cannot know it. You can only guess, and production systems fail on guesses.

A minimal evaluation practice has three parts:

1. A test set. Collect 50 to 200 real questions with known correct answers. Real questions from real users are much better than invented questions.
2. Retrieval metrics. For each test question, measure whether the retrieval step found the correct source material. This is separate from answer quality. Many “bad answers” are retrieval failures, not model failures.
3. Continuous monitoring. Quality changes over time. Documents change, users change, and models change. Run the test set on a schedule and watch the trend.

Start this practice on the first day, with the simplest lane. The test set that you build for Lane 1 remains useful in every other lane. It also gives you the data to know when a lane change is necessary, which brings us to the final section.

How to Select Your Lane

Four questions identify the correct starting lane:

1. How large is the corpus? Hundreds of pages: start in Lane 1. More: continue to question 2.
2. Can one passage answer most questions? Yes: Lane 2 is your lane. No: continue to question 3.
3. Are the answers in the relations between entities? Yes: consider Lane 3, and start with a small graph. No: continue to question 4.
4. Does the work span many sources or systems, with quality more important than speed? Yes: Lane 4, with an evaluation harness from the first day.

Two more rules apply in every case. Start with the simplest lane that answers your questions; change lanes only when the questions make it necessary. And measure retrieval quality from the first week; the measurement tells you when a change is necessary.

One more note: the lanes combine. A real system often uses Lane 2 for routine questions and Lane 4 for complex tasks. The lanes are options that you can combine. They are not exclusive commitments.

Where We Can Help

Blackburn Labs designs and builds AI systems for companies that need answers from their own data. We have implemented every lane in this guide on production systems. We can help you select a lane, build the pipeline, and put the evaluation practice in place.

If your team is at the start of this decision, we offer a short working session: we map two or three of your real workflows onto these lanes and give you a direct recommendation. Contact us to set it up.