A previous post described the retriever we run on almost every question: a hybrid search that fuses meaning-based matching with exact keyword matching, reranks the result, and survives a citation to a specific subsection. That default is good, and most questions never need anything more. This post is about the questions that do.
Some questions are worded so vaguely that the words in the question barely overlap the words in the answer. Some are broad enough that a single phrasing only finds one corner of the answer. Some cannot be answered in one lookup at all, because the second thing you need to find depends on what the first lookup returned. And some retrievals come back uncertain, a pile of passages where half are on point and half are noise, and the honest move is to grade them before trusting them.
For each of those shapes we have a different move layered on top of the default, and none of them replace hybrid search. They decide how the query reaches it, how many times, and what happens to what comes back. This post walks through the five, grounded in what our code actually does, including the honest part: which fire by default, and which are opt-in or reserved for the deeper tier. There are two small snippets, both simplified for readability.
The dispatch: usually the default, sometimes not
Every query is assigned a retrieval strategy before it searches. Most of the time that strategy is just the tier default, and for our fast tier the default is the adaptive one described later, while for the deep tier it is the multi-hop one. A few query shapes get promoted to a different strategy automatically, and a caller can force one explicitly.
# Simplified from our retrieve_node strategy dispatch.
strategy = forced_override or tier.default_strategy # most queries: the tier default
if query_looks_like_a_comparison and tier.allows_multi_hop:
strategy = "multi_hop" # "compare X and Y" earns a second hop
chunks = await run(strategy, query) # self_rag | multi_hop | hyde | multi_query | crag | hybrid
In plain terms: there is one place that picks the retriever, it leans on the tier's default, and it upgrades a query to a heavier strategy only when the query's shape justifies the cost.
One honest caveat up front, because it shapes everything below. Our flagship path, an unscoped US research question with no document pinned to it, has its own optimized retriever that already folds in query rewriting, and it returns before this dispatch runs. So the strategies below apply most visibly when you are talking to a pinned document, when a query is a genuine multi-part comparison, or in the deep tier where they run as extra recall passes. I will say which is which as we go, rather than imply all five fire on every question.
Drawn out, the decision that reaches the six-way dispatch looks like this.
Two of those branches are worth naming precisely. The comparison upgrade is not the model's whim: an upstream analyzer flags a query as decomposable, and the upgrade to multi-hop only fires when the tier is actually allowed to run it, so a fast-tier comparison stays on its cheaper default rather than paying for extra hops. And the forced override at the top of the pick is how the system recovers when an adaptive strategy grades every passage out and returns nothing: a no-results retry re-runs the query on plain hybrid, which always returns its top-k, instead of failing empty.
HyDE: answer the question first, then go find it
HyDE (Gao et al., 2022) is the technique this section borrows its name from.
A question and its answer are often written in different vocabularies. A user asks "what happens if my vendor gets breached and sits on it." The controlling language says "the covered entity shall notify affected consumers no later than the period specified." Those two sentences mean the same thing and share almost no words, and meaning-based search leans on shared words more than we would like.
HyDE closes that gap by writing the answer before searching for it. We take the question, ask a small fast model to draft a short hypothetical passage that would answer it, and then search with that passage instead of the bare question. The draft is never shown to anyone and never treated as true. It exists only as better bait: a passage written in the register of a real source lands near real sources in vector space.
The part worth grounding is that our generator picks the register deliberately. Before it drafts, it decides what kind of source would actually answer this question. A dispute-shaped question gets drafted as court-opinion prose, a compliance question ("what does the breach-notification rule require") as statutory or regulatory text, and a transactional question ("what should the liability cap look like") as executed-contract clause language. It does not default to sounding like a judicial opinion when the real answer lives in a statute or a contract, because that mismatch is exactly what would pull the wrong passages.
The honest limit: HyDE is not our everyday retriever. It runs as an opt-in strategy when a caller asks for it, and as one of the extra recall passes the deep tier merges into the candidate pool. It is a tool we reach for when vocabulary mismatch is the suspected problem, not a tax we levy on every query.
Multi-query: ask the same thing several ways
A single phrasing of a broad question only ever probes one region of the corpus. "What is the test for ineffective lawyering" will not, on its own, reliably find the passages that say "ineffective assistance of counsel," because the retriever is anchored to the words you chose.
Multi-query fixes this by not trusting any one phrasing. We ask a small model to rewrite the question into a few alternate phrasings that preserve the legal meaning but vary the vocabulary, then search all of them in parallel and merge. The rewrites deliberately swap in synonyms for doctrines and add statutory citations the user did not think to include, so the net cast is wider than any single phrasing could manage.
# Simplified from our US multi-query fan-out.
rewrites = await paraphrase(query, n=N_REWRITES) # a few alternate phrasings, same legal meaning
queries = [query, *rewrites]
results = await gather(retrieve(q) for q in queries) # search all of them in parallel
chunks = dedupe(flatten(results)) # merge, drop duplicates, then rerank
In plain terms: we generate a handful of ways to ask the question, search each, pool the results, remove the duplicates, and let the reranker sort the union. The rewriting fails safe. If the model call errors or times out, the fan-out quietly collapses back to searching the original question alone, and the results are cached so a follow-up on the same question does not pay for the rewrites twice.
This is the one alternate technique that is genuinely on the everyday path. On our main US research route, the fan-out is built in, so a broad question is already being asked several ways before you see an answer. The deep tier runs a second, generic version of the same idea as an extra recall pass on top of whatever base strategy it chose.
Multi-hop: when the second search depends on the first
Some questions cannot be satisfied by one search, no matter how well phrased, because they are really two questions stacked, and you cannot retrieve the second part until the first has told you what to look for. A comparison across two provisions, or a question whose answer hinges on a fact you have to surface first, has this shape.
Multi-hop handles it by searching in rounds. It retrieves an initial set, then a model reads what came back and decides whether the evidence is sufficient. If it is not, the same model names what is missing and proposes a follow-up query, and the next round searches for that. The rounds accumulate, deduplicating across hops, until the evidence is judged sufficient or the hop budget runs out.
Two things about our implementation differ from the textbook version. The reasoning between hops is done over the retrieved passages, not over an intermediate answer the model wrote to itself, so the loop is gathering evidence rather than talking itself into a conclusion. And the early stop is deliberately aggressive: once enough passages are in hand, it stops, so a question that looks multi-hop but resolves quickly often finishes in a single round despite the budget allowing more. That is a cost-and-latency choice, not an accident, and it is why multi-hop is the default only on the deep tier and for queries whose shape clearly signals a comparison.
Self-RAG: retrieve, then check whether you retrieved enough
The name comes from Self-RAG (Asai et al., 2023); the corrective grading layer draws on Corrective RAG (Yan et al., 2024).
Self-RAG is the adaptive strategy, and it is the default on our fast tier for the non-flagship paths. In its full form it can decide a query needs no retrieval at all, answer, critique its own answer, and regenerate. Here is the honest version of what actually runs in retrieval.
We keep the "should I even retrieve" assessment, and it also pulls out any explicit count the user asked for, so "give me ten cases" adjusts how many results we aim for. But in production we run it configured to always retrieve, because a separate upstream gate has already peeled off the pure chitchat. The live value of self-RAG for us is not skipping retrieval. It is the check afterward.
After the first search, if what came back is too few passages or too weak on average, the retrieval is treated as insufficient. When it is, a model does a gap analysis over the passages in hand, names what is missing, proposes a follow-up query, and we search again and merge. So self-RAG, as we run it, is really "search, notice when the result is thin, and go back for a second pass aimed at the gap," rather than the full retrieve-critique-regenerate loop the name implies. I would rather describe the pass we actually run than the paper we borrowed the name from.
Corrective RAG: grade the pile before you trust it
The other four strategies decide how to search. Corrective RAG decides what to do with the pile of passages after the search, and it runs as a correction layer on top of whatever base strategy produced them, on both tiers.
It scores each retrieved passage for relevance to the question, then classifies the whole set: mostly good, mixed, partial, or mostly noise. A mostly-good set passes through untouched, and a mixed set gets its irrelevant passages filtered out so the model that writes the answer is not reading padding. A partial or weak set is the interesting case: rather than throwing the passages away, it keeps the relevant remainder and flags that the corpus alone may not be enough, which is what lets an answer be supplemented from a live web search and mixed in rather than replaced. Even on a weak set it preserves the best few passages instead of returning nothing, because "here is the closest thing I found" beats an empty result a user cannot see into.
The one place it steps back is scoped work. When you have pinned a specific document or a specific case, the scope is the point, and grading passages out of your own document to make room for something else would be wrong, so the corrective layer stands down for pinned retrieval.
Default versus opt-in, stated plainly
The honesty in one place, because a buyer should know which is marketing and which is the running system. Multi-query is the most live: on the main US research path the fan-out is built in, so broad questions are already asked several ways by default. Corrective grading runs as a layer on both tiers. Self-RAG is the fast-tier default for pinned-document and scoped work, but the part that earns its keep is the gap-driven second search, not the skip-retrieval headline, which we keep switched off. Multi-hop is the deep-tier default and the automatic pick for comparison questions. HyDE is the most opt-in of the five.
The point underneath all of it: the retriever is a decision made per question, not a fixed pipe. One blend is not correct for a vague question, a broad question, a two-part question, and a clean lookup all at once, so we do not pretend it is.
Test it yourself: does the retrieval adapt, or stay shallow
You do not need our internals to see whether a tool varies its retrieval or runs one shallow pass on everything. You need two questions and a couple of minutes.
First, the vague-vocabulary question. Ask something real but worded in plain language, deliberately avoiding the term of art: "if someone's lawyer basically slept through the trial, what can they do about the conviction later," without ever saying the doctrine's name. A tool that reformulates before searching finds the on-point authority anyway. The tell of a shallow one: it only finds sources that happen to reuse your casual words, and misses the doctrine that answers the question.
Second, the two-part question. Ask something whose second half depends on the first: "which of these two provisions imposes the shorter deadline, and what is the consequence of missing it." A tool that can search in rounds surfaces both provisions and the consequence. The tell of a single-pass one: it answers confidently about one provision and ignores the other, or gives you the deadlines and forgets the consequence.
Then ask the same tool a plain lookup, "quote the definition of this defined term," and watch whether it treats all three the same. A system that runs one retriever for every question will look identical on all three, and visibly weakest on the first two. A system that varies its approach will look like it worked harder on the hard ones, because it did.
And the questions to ask the vendor
The prompts test the product. These test whether the people behind it thought about the problem.
Do you use one retrieval method for every question, or does the approach change with the question? A vendor who cannot describe more than one is describing a ceiling.
For a vague question, do you reformulate before searching, or search the user's exact words? Searching the literal words is where vocabulary mismatch quietly buries the right authority.
For a multi-part question, can retrieval run in more than one pass, and when does it stop? "One pass, always" is a real answer, and it tells you exactly where the tool will thin out.
After retrieval, do you grade the passages before answering, or feed the model whatever came back? Grounding written over noise is grounding you cannot trust.
A vendor who answers these concretely is doing real retrieval engineering. A vendor who says "we use RAG" is naming a category, not a system, and the difference is the whole game.
