A legal-AI product is not one feature. It is a stack of them that all have to be right at the same time: retrieval, the RAG pipeline, the agentic tool loop, the prompts, jurisdiction routing, doc-mode chat over your uploaded files, the handoffs to compare and redline and matrix, and the verifier that grades the final answer. Change any one of those and you can move any of the others. A prompt tweak that sharpens statute answers can dull a follow-up question, and a routing improvement that helps "summarize this contract" can send "rewrite this transcript, do not summarize it" to the wrong engine.
The honest problem for a small team is that you cannot re-test that surface by hand before every change. By the time you have manually clicked through jurisdiction routing, doc-mode, handoffs, and a dozen corpus questions, you have spent an afternoon and still checked a fraction of what matters. So we built the checking. This post is how it works, what it costs, and the one rule that keeps a fixed problem from ever coming back, followed by a short test you can run against any legal-AI tool and the questions to ask its vendor.
Two layers, because the failures are two different shapes
The failures we guard against come in two shapes, so we catch them with two different tools.
The first is a routing mistake: the user gives a clear instruction and the system sends it to the wrong engine. "Rewrite this deposition in the third person, this is a complete recitation and not a summary" is a targeted instruction that must be obeyed inline, and turning it into a whole-document summary is the exact outcome the user did not ask for. That failure is deterministic; it happens in the classifier that reads the prompt, before any retrieval or generation runs, so it needs no live data or spend to reproduce, and we test it at scale for free.
The second is a quality regression in a real answer: the routing was right, retrieval ran, the model wrote something, and it is worse than last week, whether a missing element of a claim, a source leaked from the wrong jurisdiction, or a handoff chip that stopped appearing. You cannot see that by reading the classifier; you have to run the whole product against real data and read the answer, which we do on a live server, against production matters, with a strong evaluator model grading every reply.
Here is the whole gate a change passes through before it ships.
Layer one: a free, deterministic routing sweep
The router that decides what a prompt means is the highest-leverage place a legal answer can go quietly wrong. It reads an instruction and picks an engine: answer this inline, summarize the whole document, extract a table, run a full review, build a chronology. When it picks correctly, everything downstream has a chance; when it picks the summarizer for a prompt that said "do not summarize," the rest of the pipeline executes flawlessly on the wrong task.
Because that decision runs before retrieval and before any model call in the main path, we can test it without a server and without spend. Our routing stress module builds a corpus of labeled prompts and runs each one through the document-task classifier, which is pure pattern work that resolves in milliseconds. The corpus is generated, not hand-listed: a set of prompt templates, each fanned across 25 document nouns (contract, transcript, NDA, lease, deposition, motion, and so on), which expands into more than 1,300 distinct prompts across roughly two dozen labeled categories, and each category carries the route it should resolve to, chosen from first principles rather than from what the code does today.
The headline number is deliberately narrow. It is not overall accuracy. It is the count of severe misroutes: a prompt whose intent is targeted (rewrite, translate, redact, fill in, quote verbatim, reply, answer a specific question) that got hijacked into a whole-document engine like summarize, extract, matrix, or review. That is the family that turns a user's precise instruction into a different task, and it is the one we drive toward zero. The reverse mistake, a genuine whole-document request that falls back to a plain targeted answer, is tracked too but treated as minor: the user still gets an answer to their question.
The categories are chosen to attack the classifier where it is weakest. There is an explicit negation set ("do not summarize this lease, quote its notice provision"), because a stray "not" is one of the hardest things for a fast matcher to honor. There is an output-shape set ("what are the payment terms, put it in a table"), because asking for a table is a formatting preference, not a request to run the extraction engine. There is a set of targeted questions that merely contain a task keyword ("what does the summary section of this brief say"), because the word "summary" in a question is not a request to summarize. And there are short follow-up fragments ("the fees," "both," "more like that"), which have to be read in context and default to targeted.
Because this sweep is free and deterministic, it runs as often as we like while the router is being changed. It is the fast inner loop: edit a guard, rerun in seconds, watch the severe count.
The rule that makes the fix permanent
The classifier has a language-model authority that can override the fast pattern match on genuinely ambiguous prompts, and like anything model-driven it is occasionally surprising. So there is a second, tiny probe: a curated set of content-generation prompts where the pattern rule says targeted but the model might reach for a whole-document route, run through the live resolver at the cost of about one model call each.
Here is the discipline that matters more than any single number. When that probe confirms a real override mistake, we do not just note it. We convert it into a deterministic guard in the router itself, so the pattern layer now settles that case before the model is ever consulted. From then on it is caught for free, forever, by layer one. The fragile, expensive judgment is replaced by a cheap, permanent one, and the same regression cannot return, because there is now a test that would fail the instant it tried. This is the core of how a small team stays ahead of a large surface: you do not out-click the problem, you make each problem you solve solve itself for good.
Layer two: a live harness against production data
The routing sweep proves the system aims correctly, but it cannot prove the system answers well, because a real answer depends on live retrieval, the actual corpus, the prompts, and the model working together over the wire. So the second layer drives the real product.
It hits the exact endpoint the web UI uses, the streaming chat path, so a run exercises everything a user touches: document-task routing, pinned-source cards, citation correction, and the handoff chips that offer compare or redline or matrix. It runs 61 scenarios across 12 categories, chosen to cover the surface rather than to flatter it. There are 15 US corpus and case-law questions that also assert the jurisdiction and legal area the query resolved to, 8 doc-mode conversations against real pinned contracts, and 6 handoff scenarios that check the compare, version-diff, matrix, and draft offers actually appear. The rest span agentic web-search turns, toggle pairs that ask the same question with every research capability on and then off, mention pairs that ask a question with and without a document attached to prove the attachment changes the answer, plus dates, follow-ups, meta and clarify intents, cross-cutting statute lookups, draft mentions, and skill selections.
Each scenario runs two kinds of check.
The first is deterministic and cheap. It asserts the request did not error, the answer clears a minimum length, retrieval returned enough sources where sources are expected, the answer contains the substrings a correct reply must contain, and, for US corpus turns, that the jurisdiction and area resolved as expected. It also runs a jurisdiction leak guard: US-scoped answers must not draw sources carrying non-US fingerprints, and the fingerprints are kept precise so a genuine US case name never trips a false alarm. Doc-mode grounding is asserted on the content of the answer, not on a source count, because a "summarize this document" turn reads the whole file and emits no source cards by design, and a naive count check would punish correct behavior.
The second check is qualitative. A strong evaluator model reads each answer as a demanding senior attorney would and scores it, harshly, for legal accuracy, invented authority, missing caveats, and whether it actually answered the question. When retrieved sources exist, the evaluator verifies claims against those sources rather than its own training memory, so it does not false-flag a correct answer about a recent statute or release simply because the fact postdates what the evaluator happens to know. When an answer is a short handoff stub whose real output is a downloadable redline or tracked-changes card, we skip the qualitative score, because grading a one-line "I will draft that for you" as if it were the whole deliverable would produce a meaningless number.
Why it is a live harness, and why that is the right call
This harness is intentionally not a continuous-integration test. It needs a running server, live production data, and real model spend, so it never runs automatically in CI. That is a deliberate tradeoff, and for this domain it is the correct one.
A legal answer is only as good as the corpus behind it and the retrieval that reaches into it, and a mocked corpus with a stubbed model would test the plumbing while telling you nothing about whether the product gives a lawyer a defensible answer. The failures that matter here (a thin retrieval, a leaked jurisdiction, a dulled follow-up, a handoff that stopped firing) only appear when the real system runs end to end against real documents, so we accept the cost and run the real thing.
We contain that cost with discipline rather than by faking the inputs. The harness runs against an internal account and a curated set of test matters (an MSA and a DPA, a Delaware startup memo, a mutual NDA), never real customer data, because it creates chat rows and spends budget. It runs with bounded concurrency, so a full sweep is minutes, not an afternoon. And it is the right layer to spend real money on, precisely because layer one already caught, for free, everything that could be caught for free.
Diff against the last run, and treat drops as regressions
A single run is a snapshot; the signal is the comparison.
Every run is archived: a timestamped report and a structured JSON of every answer, its severity, its evaluator score, its source count and jurisdiction, plus stable pointers to the latest run. The archive stores each answer with the top of its sources, so a past run can be re-scored later without paying to run it live again.
Before a rollout we run the harness and diff it against the last saved run. A new failure is a regression. A large drop in evaluator scores in a category is a regression, even when nothing technically errored, because a quietly worse answer is exactly the kind of decay a legal buyer cannot afford and a human spot-check would miss. A handoff chip that used to appear and now does not is a regression. We treat those as blocking and understand them before shipping, rather than discovering them from a customer. The point of diffing rather than eyeballing is that it turns "it feels the same" into a number you can defend: the product does not have to look unchanged, it has to be measured unchanged, or better, on a fixed set of questions before it goes out.
Put the two layers together and the strategy is simple: catch what is deterministic for free and make every catch permanent, spend real money only on what genuinely requires the live system and spend it against real data, and let each run's diff against the last, not intuition, decide whether a change ships. None of this requires a large team; it requires refusing to solve the same problem twice, and running the real product against real data before you trust it in front of a lawyer.
Test any legal-AI tool yourself
You do not need our harness to pressure-test a tool. You need a few adversarial prompts and five minutes.
-
The instruction-obedience test. Paste a document and ask: "Rewrite this in the third person and keep every detail, this is a complete recitation and not a summary." A strong tool rewrites the whole thing in the third person. The tell of a weak one: it hands you a short summary, which is the opposite of what you asked for.
-
The negation test. Ask: "Do not summarize this contract, just quote its notice provision word for word." A strong tool quotes the one provision. The tell: it summarizes the whole document anyway, because a stray "not" slipped past its router.
-
The formatting-is-not-a-task test. Ask a narrow question and add "put it in a table," for example "what is the termination date, put it in a table." A strong tool answers the narrow question and formats it. The tell: it launches a full multi-column extraction over the entire document because it saw the word "table."
-
The same-question-twice test. Ask a question with a document attached, then ask the same question with nothing attached. A strong tool gives a specific, document-grounded answer in the first case and a general one in the second. The tell: the two answers are identical, which means the attachment never actually changed anything.
If a tool passes these, its routing is doing real work. If it fails, it will fail the same way on your real documents.
And the questions to ask the vendor
The prompts test the product. These questions test whether the team behind it has built a way to keep it from regressing.
-
How do you regression-test before a release, and what exactly do you run? A real answer names a suite and a gate. A vague one names a vibe.
-
Do you test your prompt router separately from your generation, and how many prompts do you run through it? Routing and answering fail differently, and a team that tests them together is testing neither cleanly.
-
When you find a misroute, does it become a permanent test, or do you just patch it? This is the one that matters most. A patch fixes today. A permanent test fixes forever.
-
Do you evaluate real answers against real data before shipping, or only against mocks? For legal work, a mocked corpus proves the plumbing and nothing about the answer.
-
Do you compare each release to the last one, and what counts as a regression? If the only definition of "no worse" is that nothing crashed, they are not watching for the failure that actually costs a lawyer: the answer that got quietly worse.
A vendor who answers all five concretely has built the discipline. A vendor who cannot say whether a fixed mistake stays fixed is telling you it will happen to you next.
