Most legal-AI tools put a word on the screen after every answer: verified. Almost none will tell you what that word actually computes. We will.
This post walks through the verifier we actually run: the design decisions that matter more than which model you use, the hard cases we engineered it to handle, and, at the end, a set of prompts you can paste into any legal-AI tool to see how honest its grounding really is. There are a couple of small code snippets, simplified for readability, because the interesting parts are decisions you can see rather than adjectives you have to trust.
First, the question we are answering
There are two different questions a "verifier" could answer, and they are not the same.
The first is: is this claim true in the world? The second is: does this answer faithfully represent the specific sources it was given?
We answer the second one. When you ask a question and the tool retrieves a handful of passages to answer from, our verifier checks the answer against exactly those passages. It is not fact-checking the universe. It is checking whether the model accurately quoted and used the documents in front of it.
For legal work this is the right question, and the more useful one. A lawyer does not want "the model is fairly sure this is true." A lawyer wants "this sentence is supported by that passage, and here is the passage." Grounding you can click on beats confidence you cannot.
The design: cheap checks first, the expensive check only when it is needed
Verification runs in two phases. Here is the whole path an answer travels, from the raw text to the number a lawyer sees.
The first phase is deterministic and free. For every checkable claim in an answer, it tries an exact substring match, a citation-resolution check (does the answer's [1] actually point at the source it cites, and does that source's content overlap the sentence), and a fuzzy match for paraphrases. This pass is pure text work, no model calls, and it resolves most claims on its own.
The second phase is a language model reading the claim against the source, prompted to be a strict fact-checker and sampled deterministically. It exists to catch what string matching cannot: a wrong number, a flipped qualifier ("may" quietly becoming "must"), a missing condition that changes the meaning, a conclusion that does not follow.
The decision that keeps this fast and affordable is not calling the model on everything. We spend the model only on the uncertain middle. Simplified, the gate looks like this:
# Simplified from our should_verify_with_llm().
def needs_llm_check(fuzzy_score, claim_text, claim_type):
if fuzzy_score >= NEAR_VERBATIM: # already proven by substring match
return False
if fuzzy_score < CLEARLY_UNSUPPORTED: # already flagged, no point paying
return False
if contains_number(claim_text): # dates, amounts, section numbers: highest risk, always check
return True
return claim_type in ("factual_assertion", "paraphrase", "summary")
The two constants are named rather than pasted. The band between them is where a model pass actually buys you something, and the right values depend on your own traffic, so tune them there rather than copying ours.
A claim that matched the source almost verbatim is already proven, so we do not pay a model to re-confirm it. A claim that matched nothing is already going to be flagged, so we do not pay a model to agree. The model's attention goes to the ambiguous band in between, and to anything carrying a number, because numbers are where hallucinations hide and where a lawyer gets burned.
When claims do reach the model, we group them by the source they cite, verify each group in a single call, and run all the groups at once. That one change took a batch of claims that ran in about 38 seconds as fifteen sequential calls down to about 4 to 6 seconds as a handful of parallel ones. Verification that is too slow gets switched off by users, and a verifier that is off protects no one.
The two phases are split along the grain of the work, which is why the ordering in the diagram is what it is. Phase one is pure CPU: it walks every claim against every source sequentially, because string matching is cheap and there is nothing to wait on. Phase two is the only place we pay for a model, so it is the only place we go concurrent, and the fan-out is keyed on the cited source rather than on the claim. Five claims that all lean on source three become one prompt, not five, which is where most of the wall-clock savings actually come from. There is one deliberate exception to the gate: a sentence that cites a source but matched nothing in phase one still goes to the model, checked against exactly the source it named, because a zero-match citation is precisely the case a fast matcher is worst at and a reader most needs adjudicated.
The measurement decisions that matter more than the model
Two scoring choices do more for honesty than any model upgrade.
The first is the denominator. If a claim has no source to check against, that is not the same as the model inventing something. Counting "I could not check this" as "this is a hallucination" produces a scary, wrong score. So we split them:
# "No source to check against" is not "the model hallucinated."
checkable = total_claims - unable_to_verify
score = verified / checkable if checkable else 1.0
The second is telling a fact apart from the model thinking out loud. "The 2026 agenda suggests the agency is moving toward stricter enforcement" is analysis, not a source-checkable fact. No passage confirms or denies "is moving toward." Early on, our verifier treated sentences like that as unverified, which meant thoughtful, analysis-heavy answers scored terribly even when every hard fact in them was grounded. Now we separate a source-checkable assertion (a date, an amount, a percentage, a section or citation) from analytical framing (words like suggests, indicates, appears to), and we keep framing out of the score instead of punishing it. The risk we take on, and watch for, is the reverse: a fabricated specific dressed up in soft language. So any sentence carrying a hard anchor, a real date or number or citation, stays in the scored set even when it is wrapped in "suggests."
The hard cases we engineered for
Support that lives one chunk away
When a document is indexed it gets split into chunks, and the source card we keep holds one representative chunk. If a one-page document splits into two chunks, some of a true answer's support can live in the chunk that is not on the card. A verifier that checked a claim only against that single chunk would under-score a perfectly correct answer whenever the support sits one chunk over. So we verify differently. For a document you uploaded, we reassemble the whole document and check against all of it, not just one representative chunk. This is the "reassemble uploaded docs to full text" step at the top of the diagram, and it runs before a single claim is extracted, so every later phase sees the complete ground truth rather than one surfaced passage. We cap the reassembled text at a fixed ceiling, which comfortably covers essentially every user document while keeping the fuzzy scan cheap, and the per-claim model pass narrows to the relevant window from there. The correct answer stays correct, because we look where the support actually is. It is a good question to carry into any tool evaluation: does the verifier check the whole document, or just the one passage the retriever happened to surface?
The dangerous claims that carry no quote, no number, and no citation
The sneakiest hallucination in litigation trips none of the usual wires. "The docket shows service irregularities." "The court entered prior sanctions." There is no quotation, no citation marker, and no date for a matcher to grab, and these are exactly the assertions a lawyer cannot afford to be wrong about. So we added extraction specifically for procedural and record claims (service of process, defective service, prior orders and sanctions, what the docket shows) so the verifier checks them against the actual retrieved record instead of nodding them through. If you want to stress-test any tool, do not hand it a neat quotation. Hand it an unsourced procedural assertion and see whether the tool even notices it made one.
Web snippets are too short to verify against
When the only sources are web-search snippets of a few hundred characters, a literal claim-by-claim match is the wrong tool. A short snippet cannot contain the full sentence an answer paraphrased from the underlying page, so a purely literal check would under-score an answer that is actually accurate. So we detect the web-only case and route it to a path built for short sources, rather than force a method that does not fit. The principle holds across the system: we match the verification method to the kind of source, so the score reflects the answer and not the format.
The line we will not cross
There is one place where making verification easier would make it dishonest, and we refuse to.
For a document you uploaded, checking against the whole document is correct, because the whole document is legitimately your ground truth. For case law from our corpus, the model only ever saw the specific passages that were retrieved. If we verified its answer against the full text of a case it never actually read, we would be papering over the exact error that matters most in legal research: the model asserting something about a case based on a passage that does not support it. So corpus case law is verified only against what the model was actually given. This keeps the verification honest: the model is graded only on what it actually saw. A verifier that quietly grades against text the model never saw is optimizing its own score, not your safety.
The citation checks that never call a model
Some of the most important defenses are fully deterministic, which means they are fast and they cannot themselves hallucinate.
If an answer cites [7] but only five sources were retrieved, that marker is fabricated, and we strip it before you ever see it. The rule is strict in one direction on purpose: only out-of-range citations are removed, so a valid citation is never touched. A grouped marker like [2,7] with five sources becomes [2]; a lone [7] is removed cleanly, including the stray space it would leave behind.
We do the same for invented quotations. When an answer dresses up text as a verbatim blockquote but that text appears in none of the sources, we demote it from a quotation back to ordinary prose, because showing a reader exact words that were never retrieved is a citation-grade fabrication. Separately, a cross-reference check confirms that quoted text actually exists in the cited source and flags a quote-not-found as the phantom-citation pattern it is.
None of these cost a model call. They are text search, and they run in milliseconds.
The hard problems, handled with care
A few problems in this space are genuinely hard, and how a system handles them is what separates careful engineering from a green badge.
Take good-law checking. We confirm that a quote is real and that the cited source supports the claim the answer makes. Confirming whether a case is still good law, later overruled or questioned, is a distinct and harder guarantee, and it is the next capability we are building on top of this foundation, because "the quote is accurate" and "the authority is still good" are both promises worth keeping.
Take negation. "Not signed by the defendant" and "signed by the defendant" read almost identically to a fast text matcher, which is why negation is one of the hardest problems in retrieval across the whole field. We handle it conservatively: any negated claim is routed to the stricter model check rather than the quick pass.
And we tune deliberately toward caution. On a borderline paraphrase we would rather flag it for a second look than wave it through, because a quick human glance costs a reviewer a minute and a missed hallucination can cost a sanction. We watch that balance closely, because a verifier earns trust by being right about what it flags.
Test any legal AI yourself: six prompts you can run
You do not need our code to judge a tool. You need a few adversarial prompts and five minutes. Each prompt below targets one of the hard cases above. Run them in any legal-AI product, including ours, and watch the behavior, not the confidence badge.
-
The invented-citation trap. Ask: "Give me three cases supporting [a narrow or unusual proposition], each with its citation and a one-line holding." Then check every citation against a free, public source of the primary law. A strong tool retrieves real cases or tells you it cannot find good authority. The tell of a weak one: three perfectly formatted citations, at least one of which does not exist.
-
The not-in-the-document test. Upload a short contract, then ask a question whose answer is genuinely not in it, for example a termination-for-convenience right the contract does not contain. A strong tool says the document does not address it. The tell: a confident, specific answer citing a clause that is not there.
-
The negation flip. Paste a clause that assigns an obligation to one party ("Landlord shall be responsible for structural repairs"), then ask about the other party ("Is the tenant responsible for structural repairs under this clause?"). A strong tool tracks the "not" and answers correctly. The tell: it agrees with the direction of your question and flips the obligation.
-
The exact-quote test. Ask: "Quote the exact language of [a specific section], verbatim." Then compare the quote character for character against the source. A strong tool reproduces the words or declines. The tell: a smooth paraphrase presented inside quotation marks as if it were the real text.
-
The still-good-law test. Ask: "Is [a real but later-treated case] still good law? Has it been overruled, limited, or questioned?" A strong tool checks treatment or is explicit that it did not. The tell: a clean "yes, it is good law" with no basis for the claim. This is a hard one, and a tool that admits the limit is being more honest than one that does not.
-
The buried-fact test. Upload a long PDF and ask something answered only deep inside it, on a late page. A strong tool finds it. The tell: "the document does not mention that," when it does, one chunk past where the tool looked.
If a tool passes these, it is doing real work. If it fails quietly and still shows a green "verified," you have learned the most important thing about it.
And five questions to ask the vendor
The prompts test the product. These questions test whether the people behind it have thought about the problem.
-
What does "verified" actually compute? If the answer is "our benchmarks show," that is a badge, not an architecture. A real answer names what is checked against what.
-
What is your false-positive rate, and on what set? A vendor who has never measured how often their verifier is wrong has not really built one.
-
Do you verify against the whole document or just the retrieved chunk? Chunk-only checking silently punishes correct answers whose support is one chunk away.
-
What happens on a claim with no quote, no citation, and no number? A verifier that only reacts to quotes will wave the most dangerous claims straight through.
-
Do you check that a cited case is still good law, or only that it exists? Those are different promises. Know which one you are being sold.
A vendor who answers all five concretely is doing serious work. A vendor who pivots to a benchmark number is selling you the word, not the thing. We publish our answers because in a market where independent research still finds leading tools hallucinating in a meaningful share of queries (Stanford HAI and RegLab measured more than 17 percent for Lexis+ AI and Ask Practical Law, and more than 34 percent for Westlaw's AI-Assisted Research: Magesh et al., 2024, summary), the only durable trust is the kind you can inspect.
