A wrong legal answer rarely looks wrong. It looks fluent, it cites something, and it carries the same calm tone as a correct one. That is the whole problem. The surface confidence of a language model tells you almost nothing about whether the answer is reliable, because the model is equally articulate when it is right and when it is bluffing.
So we stopped trusting the model's own certainty and started measuring reliability directly. This post is about three signals we compute on the deep tier to estimate whether an answer can be trusted, and how each one pulls the displayed confidence down when the system is genuinely unsure. None of them ask the model "are you sure?" They test it instead. There are two small snippets, simplified for readability, because the decisions are more interesting than the code around them.
Here is how the three signals feed the one number a lawyer actually sees.
The shape of that diagram is the whole thesis: many independent measurements flow into one calibration step, and that step starts from doubt and only climbs when the measurements agree.
A note before we start: this is a different job from our citation verifier, which checks whether specific sentences are supported by specific sources. That verifier answers "is this claim in the documents." The signals here answer a softer, deeper question: "even where nothing is provably false, is this answer stable enough to rely on."
Signal one: ask the same question several times and watch for disagreement
If you ask a knowledgeable person a question they actually know, they give you the same answer twice. If you ask them something they are guessing at, the answer wobbles.
We use the same tell on the model. For an answer on the deep tier, we resample: we pose the same question against the same source context several times, at a higher sampling temperature to let variation surface, and then we compare the answers. This is a well-established self-consistency technique from the research literature, and the intuition is blunt. If the model gives materially different facts each time it is asked, it is not recalling a settled answer, it is improvising, and improvisation is where hallucinations live.
We do not compare the answers as raw text, because two correct answers can be worded very differently. Instead we pull the concrete, checkable facts out of each sample, the dates, numbers, names, and holdings, and we measure how much those fact sets overlap across samples. High overlap means the model keeps landing on the same specifics. Low overlap means it is inventing different specifics each time, and that disagreement becomes a red flag that lowers the score in proportion to how far the samples drift apart.
This is not free, which is exactly why it does not run on every answer. The resampling costs roughly two to five times a single answer, and on the deep tier that works out to several extra model calls and a meaningful chunk of added latency per query. We spend that budget only where it can change the outcome. The gate looks like this.
# Simplified from should_check_consistency().
def worth_resampling(claim_count, verification_score):
if not CONSISTENCY_ENABLED:
return False
if claim_count < FEW_CLAIMS: # too little at stake to pay 2-5x
return False
if verification_score >= NEAR_CERTAIN: # already proven, do not pay again
return False
return UNCERTAIN_LOW <= verification_score < UNCERTAIN_HIGH # the doubtful middle
In plain terms: we skip resampling when there is barely anything to check, and we skip it when the answer already verified cleanly, because paying to re-confirm a proven answer buys nothing. The resampling budget goes to the uncertain middle band, the answers that are neither clearly grounded nor clearly broken, because that is the only place a second opinion can move the needle.
One engineering detail is what makes this signal trustworthy. Some reasoning-style models ignore the temperature setting you would normally use to force diverse samples, which would make every sample come back nearly identical and leave the agreement metric uninformative. So we detect that case and sample from a model that genuinely varies its output, which keeps the agreement signal meaningful instead of a number that only looks like a measurement. The principle we hold to is simple: a reliability signal is worth showing only if it is genuinely measuring something, so we audit ours to make sure it is.
There is a second deliberate choice buried in this signal. If the resampling itself fails, an API error, a timeout, a parse failure, we return a neutral result and do not penalize the answer. A failed measurement is not evidence of a bad answer. Penalizing an answer because a check could not complete would be as misleading as counting "I could not check this" as "the model lied," so we do neither.
Signal two: score every sentence for whether the sources actually support it
Claim extraction is good at catching sentences that look like claims, a date, an amount, a citation. It is worse at the sentence that asserts something real while carrying no obvious anchor to grab.
So alongside claim checking, we score groundedness sentence by sentence. We split the answer into sentences, split every source into sentences, and for each answer sentence we go looking for the best supporting sentence anywhere in the sources. A sentence that appears almost verbatim in a source scores high. A sentence that overlaps a source heavily in wording and structure scores in the middle. A sentence that finds no real match anywhere in the sources scores low and gets flagged as ungrounded. The middle score is not one measurement but a blend: an exact substring wins outright, and short of that we combine a token-overlap measure, a word-order-tolerant measure, and a best-substring measure, weighted so that no single quirk of phrasing can carry a sentence over the line by itself. Splitting into sentences is also less trivial than it sounds, because a source in a script without spaces between sentences would otherwise collapse into one giant block that matches nothing, so the splitter is built to handle those terminators too and keep the signal honest across the languages our users actually paste. The overall groundedness of an answer is the average across its sentences, and when that average falls below the bar, we treat the answer as less reliable and pull the score down in proportion to how poorly grounded it was.
The value here is coverage. The verifier can wave through a smoothly written sentence that no source supports, because it does not trip any of the usual claim patterns. Sentence-level scoring gives every sentence a number regardless of whether it looked like a claim, which is how you catch the confident aside that slipped in without a source behind it.
There is a case worth calling out because it is the honest one. When the answer has sentences but the sources contain no usable sentences to check against, we do not shrug and pass the answer. We score it as fully ungrounded, because an answer with nothing behind it is exactly the answer a lawyer most needs to be warned about. The reverse is also handled with care: a wall of exact-match text is not automatically trustworthy, so an exact match and a strong-but-partial match are scored differently rather than collapsed together.
Like resampling, groundedness scoring runs only where it can matter. It skips very short answers, it skips answers with no sources at all, and it skips answers that already verified with high confidence. It concentrates on the same doubtful middle, the answers where an extra, sentence-granular look is worth the compute.
Signal three: calibrate the number down for overconfidence, up only when independent checks agree
The last step is the one that decides what number you actually see.
Raw confidence out of a language model runs hot. The research consensus (Kadavath et al., 2022), and our own experience, is that these models overstate their own certainty, so taking their confidence at face value would systematically overpromise. Our calibration starts from a position of distrust: it discounts the raw confidence before anything else, on the assumption that the unadjusted number is too high.
Then it earns some of that back, but only through agreement. If several independent checks all point the same way, the string matcher, the model-based verifier, the consistency resampler, the groundedness scorer, that convergence is a real signal, and independent methods agreeing is much harder to fake than a single method being loud. So the calibration boosts the score when multiple methods concur, with a larger boost when three or more agree than when only two do, and then clamps the result into a sane range.
# Simplified from _calibrate_confidence().
def calibrate(raw_confidence, methods_that_agree):
# Start low: models overstate their own certainty.
score = raw_confidence * OVERCONFIDENCE_DISCOUNT
# Give it back only when independent checks converge.
if len(methods_that_agree) >= 3:
score *= STRONG_AGREEMENT_BOOST
elif len(methods_that_agree) >= 2:
score *= SOME_AGREEMENT_BOOST
return clamp(score, 0.0, 1.0)
In plain terms: doubt is the default, and confidence has to be corroborated to survive. A single method saying "very confident" is treated skeptically. Three independent methods quietly agreeing is treated as trustworthy. That is the opposite of how a raw model score behaves, where one loud pass sets the tone.
The aggregation around this method matters as much as the multipliers we are not publishing. The displayed number is an average of calibrated confidence across the answer's checkable claims, and it deliberately leaves out the claims we could not check against any source and the analytical framing that no source can confirm or deny. Earlier, folding those uncheckable pieces into the average dragged thoughtful, analysis-heavy answers down toward the floor even when every hard fact in them was solid. Keeping "I could not check this" out of the denominator is the same honesty principle as the rest of the system: absence of evidence is not evidence of a lie, and it should not be scored as one.
Why all of this lives on the deep tier
Every signal here costs real money and real time. Resampling multiplies the cost of the answer several times over. Groundedness scoring adds another pass. Calibration is cheap, but it is only meaningful when the expensive signals have run and there is agreement to measure.
Running this on every quick lookup would make the product slow and expensive for answers that did not need it. So the heavy reliability signals are reserved for the deep tier, the mode a user reaches for when the stakes are high enough to wait, and even there they fire only inside the uncertain band where a second look can change the verdict. A near-certain answer skips them. A clearly broken one is already caught upstream. The budget goes to the answers in between, the ones that look fine and might not be.
The through-line across all three is a single preference. We would rather show a lower number and be right about our own uncertainty than show a confident badge that turns out to be hollow. A tool that is never unsure is not careful, it is just not looking.
Test it yourself
You do not need our internals to check whether a legal tool knows when it does not know. You need one genuinely hard question and a few minutes.
-
The repeat test. Ask any legal-AI tool the same nuanced question three or four times in fresh sessions, something with real ambiguity, like how a specific doctrine applies to an unusual fact pattern. A tool with reliability signals gives you a stable answer or tells you the question is unsettled. The tell of a weaker one: three fluent answers that quietly assert different specifics, different cases, different numbers, each delivered with equal confidence.
-
The confidence-honesty test. Find a question the tool cannot actually answer from its sources, then watch what its confidence does. A tool that measures its own reliability should get less sure, not more. The tell: an unshakable, fully confident answer to a question it had no basis to answer.
-
The unsupported-aside test. Ask something answerable, then read the answer for the extra sentence, the confident aside with no citation behind it. A tool scoring groundedness at the sentence level should flag or hedge that sentence. The tell: a smooth, sourced answer with one unsupported claim riding along, indistinguishable in tone from the rest.
And three questions for the vendor.
Do you ever ask the model the same question more than once to check whether it agrees with itself, and what do you do when it does not? Do you score reliability at the level of individual sentences, or only for claims that look like claims? When your own reliability check fails to run, does the answer get penalized, or do you avoid punishing an answer for your infrastructure's problem?
A vendor who can answer those concretely is measuring uncertainty. A vendor who points at a single confidence badge is showing you the model's self-assessment and calling it a safety feature. The difference is the whole game.
