The default answer to "how do I know this citation is real" has become: ask another model. Hand the answer and the source to a second model, tell it to be a strict judge, and read the verdict. It is easy to build, it demos well, and it is the wrong tool for most of the job.
A fabricated quote is a decidable problem. Either the string is in the passage or it is not. When you route that question to a probabilistic system, you get a probabilistic answer to a question that had a definite one, and you introduce a failure mode that did not previously exist: a model hallucinating agreement with a hallucination. The judge reads a quote that is almost right, decides it is close enough to what the passage says, and returns "supported." Now you have two systems that are confidently wrong instead of one, and no way to tell which.
This post is a reference recipe. Four layers, cheapest first, that verify the citations on any RAG output. It is what we run in production at Vaquill on legal documents, but nothing in it is legal-specific. If your system produces text with markers pointing at retrieved passages, this applies as written. The code is simplified from ours where noted, and it is the point of the post.
The shape: four layers, model last
Read the funnel from the top. Every layer is cheaper than the one below it and narrows what reaches it. By the time you get to the model, most claims have already been decided by arithmetic and string search, in milliseconds, for nothing.
Layer 1: is the marker even pointing at a source
When a model cites inline it writes markers like [1] or [2,7] that index into the passages you retrieved.
Sometimes it writes [7] when you retrieved five.
That marker points at nothing.
It is not a low-confidence citation, it is not a judgment call, it is a number larger than the length of a list.
So the first check is arithmetic.
# Simplified from strip_fabricated_citations().
CITATION_RE = re.compile(r"( ?)(\[(\d+(?:[-,]\d+)*)\])")
def strip_fabricated_citations(text: str, source_count: int) -> tuple[str, list[int]]:
removed: list[int] = []
def rewrite(match):
lead = match.group(1) # the optional space before [N]
nums = expand_refs(match.group(3)) # "2,7" -> [2, 7]; "1-3" -> [1, 2, 3]
if not nums:
return match.group(0) # unparseable, never touch it
valid = [n for n in nums if 1 <= n <= source_count]
invalid = [n for n in nums if n < 1 or n > source_count]
if not invalid:
return match.group(0) # all in range, keep verbatim
removed.extend(invalid)
if valid:
return lead + "[" + ",".join(map(str, valid)) + "]" # [2,7] -> [2]
return "" if not_followed_by_word(match) else lead
return CITATION_RE.sub(rewrite, text), removed
The rule is strict in exactly one direction, and that is the whole design. Only an out-of-range index is treated as fabricated. An in-range citation can never be removed, because in-range is the definition of valid here, which is what makes this safe to run unconditionally on every answer with zero false positives. You are not deciding whether a citation is good. You are deciding whether it exists.
Two details worth stealing.
Grouped markers get partially cleaned rather than nuked: [2,7] against five sources becomes [2], keeping the half that was real.
And the regex captures the leading space so removing a marker does not leave a double space or, worse, weld two words together.
We deliberately do not run a document-wide whitespace pass afterward, because that collapses intentional formatting elsewhere in the text.
Clean up locally, at the site of the edit, and nothing else in the document can be damaged by your fix.
There is a useful signal falling out of this for free. Count the markers and count the fully fabricated ones, and you have a ratio. When every citation in an answer fails, that is not a marker problem, that is an answer problem, and it is a good trigger for regenerating rather than displaying.
Layer 2: is the quote actually in the passage
The second layer is the one that does the most work and the one people most often hand to a model. Do not.
The check is deliberately dumb: normalize the quote, normalize the cited passage, and ask whether one is a substring of the other.
# Simplified from _resolve_and_verify_citations().
verified = []
for cit in model_citations:
idx = int(cit.ref) - 1
if idx < 0 or idx >= len(passages):
continue # layer 1 logic, applied to structured refs
body = passages[idx]["content"]
if normalize(cit.quote) not in normalize(body):
log.info("citation_quote_unverified", ref=cit.ref, quote=cit.quote[:80])
continue # the quote was not really there
verified.append(build_citation(cit, passages[idx]))
That is the entire fabrication check. A model asked to quote verbatim will still, sometimes, produce a quote that is almost right: a word swapped, a number nudged, a phrase smoothed into better prose. "Almost right" is exactly what a judging model forgives and what a substring check does not.
The engineering that matters is not the in operator, it is normalize.
Get it wrong in one direction and you drop good citations because a PDF used a curly apostrophe and the model emitted a straight one.
Get it wrong in the other direction and you normalize so aggressively that different text starts matching.
Our normalizer, in order: NFKC unicode normalization to collapse compatibility forms and width variants, a fold map for smart quotes and dashes and non-breaking spaces that NFKC misses, whitespace collapse, trim, lowercase.
# Simplified from normalize_quote().
def normalize(text: str) -> str:
folded = unicodedata.normalize("NFKC", text)
folded = "".join(QUOTE_FOLD_MAP.get(ch, ch) for ch in folded) # " ' - nbsp
return re.sub(r"\s+", " ", folded).strip().lower()
Every transformation there is lossless with respect to meaning and lossy only with respect to typography. That is the line to hold. Fold what a typesetter changed, never fold what an author wrote. Stemming, synonym expansion, or stopword removal in a quote verifier is how you end up certifying a quote that does not exist.
The same layer has a cousin worth adding if your output supports markdown.
A blockquote is a promise that these are the exact words from a source.
When the answer dresses up invented text as a verbatim blockquote, check it against the concatenated source text and, if it is not there, strip the > so it renders as ordinary prose.
It stops claiming to be a quote.
No model, and it never touches callout boxes, which are context rather than quotation.
Layer 3: drop the failures, and downgrade when nothing survives
Layers 1 and 2 produce a set of survivors. Layer 3 is the consequence, and it is the layer teams skip.
Dropping a bad citation is table stakes: the reader never sees it, and nothing unverified is persisted. The harder rule is what happens when the survivor set is empty.
# Simplified from the matrix cell extractor's anti-fabrication contract.
verified = resolve_and_verify(parsed.citations, passages)
leaked = unverified_quote_leaked_into_answer(parsed.citations, verified, parsed.answer)
if parsed.status == "extracted" and (not verified or leaked):
status, answer = "not_found", None # support evaporated, so the claim goes with it
else:
status, answer = parsed.status, parsed.answer
If the model claimed it extracted an answer but not one of its citations survived verification, the answer is downgraded to "not found." An ungrounded answer does not get to be an answer. This is the rule that turns verification from a decoration into a gate. Without it you have a system that shows a confident claim next to a citation list you quietly emptied, which is strictly worse than showing nothing, because the reader reads the confidence and infers the support.
Note the leaked term, because it closes a real hole.
Models weave their quoted text inline into the readable answer.
So a fabricated quote can ride along inside the prose even after its citation was dropped, and the reader sees the invented words anyway.
We check whether any quote that failed verification still appears in the answer string, and downgrade if so.
The rule is asymmetric on purpose, and the asymmetry is earned. We do not throw away a correct multi-citation answer just because one redundant supporting quote was imperfect. Downgrading on any unverified citation cost us roughly 10 to 12 percent of production cells as false negatives, correct answers thrown away over a spare quote that never reached the reader, while adding no safety. The contract that survives measurement is: at least one citation must verify, and no failed quote may appear in the answer text.
Layer 4: now, and only now, spend a model
Everything above is string work, and string work has a hard ceiling. It cannot tell you that "the tenant may terminate" was rendered as "the tenant must terminate." It cannot tell you a cap of 12 million was quietly written as 1.2 million, or that a condition which changes the whole meaning got dropped from a paraphrase. Those are semantic questions with no crisp string answer, and this is where a model is genuinely the right tool. Not a fallback, not a compromise. The right tool.
The discipline is not calling it on everything.
# Simplified from should_verify_with_llm().
def needs_model_check(fuzzy_score, claim_text, claim_type):
if fuzzy_score >= NEAR_VERBATIM: # already proven by substring
return False
if fuzzy_score < CLEARLY_UNSUPPORTED: # matched nothing: already flagged
return False
if re.search(r"\d", claim_text): # dates, amounts, section numbers: always check
return True
return claim_type in ("factual_assertion", "paraphrase", "summary")
Both early returns are the same argument from opposite ends. A claim that matched its source almost verbatim is already proven, so paying a model to re-confirm it buys nothing. A claim that matched nothing is already going to be flagged, so paying a model to agree with the flag buys nothing. The model's attention goes to the ambiguous middle, and to anything carrying a number, because numbers are where hallucinations hide and where a reader gets hurt.
One deliberate exception, and it is the interesting one. A sentence that cites a specific source but matched nothing in the deterministic pass gets sent to the model anyway, checked against exactly the source it named. A zero-match citation is precisely the case a fast matcher is worst at and a reader most needs adjudicated, so it bypasses the score gate entirely.
Two operational notes for when you build this. Group claims by the source they cite and verify each group in one call rather than one call per claim, then run the groups concurrently. For us that turned a batch that ran about 38 seconds as fifteen sequential calls into 4 to 6 seconds as a handful of parallel ones. Verification that is too slow gets switched off, and a verifier that is off protects nobody. And prompt the model to be a strict fact-checker, sampled deterministically, judging the claim against one passage. You are asking it a narrow question with the evidence in front of it, not asking it to be a judge of truth in general.
Why the ordering is the design
The layers are not four independent checks you could shuffle. The order is load-bearing in three ways.
Each layer narrows the next. Range-checking first means layer 2 never wastes a lookup resolving a ref that points at nothing. Substring verification means layer 4 never spends a model call on a quote you can already prove is verbatim, or on one you can already prove is absent. By the time you reach the model, you are paying only for the genuinely undecidable, which is a small fraction of claims.
Each layer is orders of magnitude cheaper than the next. The range check is a regex scan, roughly half a millisecond for a whole answer. Citation validation runs around 5 to 10 milliseconds per citation and costs nothing. A model call is seconds and real money. Putting the expensive check first is like sorting a list by asking an oracle before checking whether it has one element.
And the cheap layers cannot themselves fail in the way the expensive one can. This is the part that matters most. A model judging a model is one probabilistic system grading another, inheriting the uncertainty of both. A substring check has no opinion, no temperature, and no bad day. You can write a unit test that proves it removes exactly the fabricated markers and nothing else, and that test will still be green next year regardless of what any model does.
So the general rule, applied to citations: use the probabilistic tool only for the question that is genuinely a language question. Everything upstream of that has a definite answer, and a definite answer deserves code.
What this does and does not buy you
Be precise about the guarantee, because a vague one is how "verified" became a badge that computes nothing.
What these four layers guarantee: every citation shown points at a real retrieved passage, every quote shown genuinely appears in the passage it cites, no claim is displayed whose support was dropped, and the semantic gap between a paraphrase and its source was adjudicated where string matching could not decide.
What they do not guarantee: that the retrieval found the right passages in the first place, or that the underlying source is correct about the world. Those are separate jobs for separate layers. Verification answers "is this answer faithful to the sources it was given," which is a narrower and far more useful question than "is this true," and it is the one you can actually build a guarantee around.
Implementation checklist
Work down this list in order. Each item is independently shippable and each one makes the next cheaper.
- Parse citation markers with one regex that handles singles, comma groups, and ranges:
[1],[2,7],[1-3]. - Range-check every index against your retrieved source count. Strip out-of-range indices only. Verify by test that an in-range citation is never modified.
- Clean up locally at the edit site. Consume the leading space, drop empty parens left behind, and never run a document-wide whitespace pass.
- Emit a fabricated-marker ratio. When every marker in an answer fails, regenerate instead of displaying.
- Write one normalizer and share it. NFKC, fold smart quotes and dashes and non-breaking spaces, collapse whitespace, lowercase. No stemming, no synonyms, no stopword removal.
- Constrain the model's output schema so a citation is structurally required and there are exactly two outcomes, answered or not found. Remove the third path where it can waffle.
- Verify every quote as a normalized literal substring of the exact passage its ref names, not of the concatenated corpus.
- Drop citations that fail. Persist only survivors.
- Downgrade the claim when zero citations survive. This is the layer that makes the rest mean something.
- Check whether a failed quote leaked into the readable answer text, and downgrade if it did.
- Keep the downgrade asymmetric: require at least one verified citation rather than requiring all of them, then measure your false-negative rate on real traffic and tune from data.
- Gate model calls on uncertainty. Skip above the near-verbatim threshold, skip below the clearly-unsupported threshold, always check anything containing a digit.
- Route zero-match-but-cited claims to the model regardless of score. It is the case string matching is worst at.
- Group model calls by cited source, then fan out concurrently. Measure the wall clock, because slow verification gets disabled.
- Split your denominator. "No source to check against" is not "the model fabricated this." Score verified over checkable, not verified over total.
- Unit test each deterministic layer against fixtures with known-fabricated markers and known-mangled quotes. These tests are permanent, unlike prompts.
The first four items take an afternoon and remove an entire class of failure. The whole list is a weekend, and at the end of it the word "verified" in your product computes something you can describe exactly to the person relying on it. That is the difference between a citation you can click and a badge you have to trust.
