What AI Legal Tools Are Actually Trained on Federal and State Case Law

A few weeks ago I watched a procurement thread where a managing partner asked four vendors the same question: "Is your model trained on all federal and state case law?" Three of them answered yes, proudly, with a number attached. Millions of cases. Billions of tokens.

The fourth said something like, "We don't really train on it, we retrieve it." That fourth vendor was the only one telling the truth, and it was the one the partner crossed off the list first.

That is the trap. The question every buyer asks about a legal AI tool ("what is it trained on?") quietly selects for the most dangerous architecture and against the safest one.

Short answer: most legal AI is not trained on case law in the way buyers picture, and the tools you can trust on purpose are not. Serious products retrieve real opinions and statutes at query time (retrieval-augmented generation) and ground every answer in a source you can open. The ones that lead with "trained on millions of cases" are describing the architecture that fabricates citations, dressed up as a selling point.

So if you are asking what legal AI tool is trained on federal and state case law, the honest answer from any tool worth using is: none of the good ones rely on that, and the ones that brag about it are describing their failure mode as if it were a feature.

The rest of this piece walks through the difference between training on case law and retrieving it, what the Stanford hallucination data actually shows, and the exact questions to ask a vendor so the demo cannot hide a fabrication.

US legal research is three distinct corpora; ask which layers a tool actually has.

TL;DR

• "Trained on case law" and "trustworthy on case law" are nearly opposite properties. A model that answers from training memory invents citations. A tool that retrieves real opinions at query time can show you the source.
• The serious incumbents (Harvey, CoCounsel) describe their research as retrieval grounded in primary authority with clickable citations, not "trained on millions of cases." The marketing-heavy vendors lean on the training claim.
• Stanford researchers found even purpose-built retrieval tools still hallucinate: roughly 17% for one major product, around 33% for another, versus about 43% for raw GPT-4. Retrieval helps a lot. It is not a cure. You still verify.
• Training also freezes the model in time. A model finalized before June 2024 still "believes" Chevron is good law after the Supreme Court overruled it. Only a retrieval and treatment layer catches that.
• The buyer question that actually predicts accuracy: "What does it retrieve from at query time, and can I open every citation it gives me?"

Part of our legal AI vendor comparison and pricing series.

Is legal AI trained on case law? Why "trained on" is the wrong question

Start with what training actually does. When a large language model trains on text, it does not store a copy of that text the way a database stores rows. It adjusts billions of numerical weights so that, given some words, it can predict plausible next words.

After enough legal opinions pass through, the model gets very good at producing text that looks like a federal opinion. It learns the cadence of a holding, the shape of a citation, the rhythm of "we therefore reverse and remand."

What it does not reliably learn is which specific citations are real. A reporter volume and page number is, to the model, just another sequence of tokens it can generate fluently.

So when you ask a purely training-based model a research question, it will happily produce Henderson v. United States, 412 F.3d 1099, with a quoted holding, and every part of that string can be fabricated while reading perfectly. The model is not lying. It is doing exactly what it was built to do: generate plausible-looking text. Plausibility is the whole danger.

This is the architecture behind the most famous legal AI disaster of the decade. In 2023, lawyers in Mata v. Avianca filed a brief in the Southern District of New York full of cases that did not exist. ChatGPT generated them from training memory, names, citations, fake quotes and all, and the court imposed a Rule 11 sanction.

We wrote that one up in detail in our piece on AI hallucinations and sanctions. The point for this post is narrower: that brief is what "trained on case law, answering from training" produces under pressure. More training data would not have fixed it. It would have made the fabrications more fluent.

So when a vendor tells you their model is trained on more case law than the competition, hear what they are actually claiming: our model has memorized the shape of more opinions, so it can generate more confident, more convincing text. None of that tells you whether a single citation it returns is real.

What the trustworthy tools actually do

The architecture that works is the boring one: retrieval-augmented generation, usually shortened to RAG. Instead of asking the model to recall an opinion from its weights, the system first searches a real corpus of actual opinions and statutes, pulls the relevant passages, and hands them to the model as context with the instruction to answer only from what was retrieved and to cite it. The model is no longer remembering. It is reading.

That single architectural choice is the difference between a tool that fabricates Henderson v. United States and one that surfaces Carpenter v. United States, 585 U.S. 296 (2018), with a link to the actual opinion you can open and read.

I walked through the full pipeline in how AI legal research works, so I will not re-litigate the mechanics here. The thing worth internalizing is that the trust does not come from the model. It comes from the corpus the model is forced to read from and the requirement that every claim trace back to it.

Watch how the people who actually build at the frontier describe themselves. Harvey and CoCounsel, the two AI-native research products most often named in the same breath as the incumbents, both frame their research as grounded in primary authority, retrieved at query time, with citations that link back to source. They do not lead with "trained on millions of cases."

Thomson Reuters, which bought Casetext for 650 million dollars in 2023 to fold CoCounsel into its stack, did not pay that to get a model with case law baked into its weights. It paid for the workflow and the connection to a verified corpus.

The whole serious field is retrieval-first. The "trained on" language lives almost entirely in the marketing of tools that want a big number to put on a slide. If you want to see how the big incumbents actually describe their training data once you read the fine print, we took apart what the Westlaw, Lexis, and OpenAI training-data terms really say.

If you want a concrete sense of what an open, retrievable primary-source corpus looks like, picture millions of US federal and state opinions you can actually pull and cite. That is the kind of thing a retrieval tool reads from. It is not something you compress into model weights and hope survives.

For related vendor / pricing / buyer-guide coverage, see Best AI Legal Research for Solo Attorneys on a Budget (2026) and Legal AI Tools for Solo Practitioners: 7 Picks That Fit a One-Person Firm.

Retrieval is not a cure, and good vendors admit it

Here is where I want to be honest, because the over-correction is its own trap. Retrieval does not get you to zero hallucinations. Anyone who tells you their RAG tool never makes things up is selling.

The cleanest evidence comes from Stanford. In a 2024 study from Stanford HAI (Magesh and co-authors, later published in the Journal of Empirical Legal Studies in 2025), researchers tested the major purpose-built legal research products and found that even retrieval-based legal tools hallucinated at meaningful rates: roughly 17% for one leading product and around 33% for another, compared to about 43% for a general GPT-4 baseline with no legal retrieval at all.

Read that twice. Retrieval roughly halved or better the error rate versus raw generation. It did not eliminate it. The corpus can be incomplete, the retriever can pull the wrong passage, the model can misread what it retrieved or stitch together two sources into a holding neither actually states.

The 2025 Vals AI legal research evaluation points the same direction from a different angle. Across that benchmark, legal-specific tools and even a general assistant using live web search landed around 80% accuracy, ahead of the human-lawyer baseline near 71%.

But the legal-specific tools pulled clearly ahead on authoritativeness, the degree to which answers were anchored in real primary authority, precisely because they retrieved from proprietary and primary sources rather than the open web. Even the general assistant only kept up on raw accuracy because it was, in effect, doing its own retrieval through search.

The thing that separated the tools was not training. It was the quality of what they retrieved from.

So the practical lesson is not "RAG good, training bad, problem solved." It is: retrieval moves you from dangerous to usable, and a verification layer plus your own professional judgment moves you from usable to filable.

ABA Formal Opinion 512 (July 2024) lands in the same place. You can use these tools, and you remain responsible for checking the output. The tool's job is to make checking fast by giving you a real source to click. That is the bar.

Try Vaquill AI on your matterBook a demo

The currency problem nobody puts on a slide

There is a second reason "trained on" fails, and it is the one I find most underrated: time.

A model's training has a cutoff. Whatever it learned, it learned as of some date, and the world keeps moving. Law moves more sharply than most domains, because a single Supreme Court decision can flip the answer to a question overnight.

Take the cleanest possible example. For forty years, Chevron v. NRDC, 467 U.S. 837 (1984) was the controlling framework for how courts reviewed agency interpretations of statutes, the famous two-step deference doctrine that shaped administrative law and ran through countless decisions interpreting things like the standard of review in 5 U.S.C. 706.

Then in June 2024, the Supreme Court decided Loper Bright v. Raimondo, 603 U.S. 369 (2024) and overruled Chevron. Forty years of doctrine, gone in one opinion.

Now think about what a model trained before June 2024 "knows." It knows Chevron deeply. It has read thousands of opinions applying it.

Ask that model how a court should review an agency's statutory interpretation and it will give you a confident, fluent, well-cited Chevron answer, and it will be flatly wrong on current law. The training did not betray it. The training is exactly the problem. The model is frozen at its cutoff, and it has no idea the ground shifted.

A retrieval tool with a current corpus and a citation-treatment layer catches this. It pulls Loper Bright, it flags Chevron as overruled, it grounds the answer in what is actually good law today. No amount of training fixes currency, because by the time you have trained, the world has already moved again.

This is also why the same logic applies to statutes and regulations. A provision like 17 U.S.C. 107 (the fair use section that is, not coincidentally, at the center of the fights over training data itself) can be amended, and a model frozen at a cutoff will recite the old text with total confidence. Pulling the live section text at query time is the only thing that keeps you current.

That live statute layer (U.S. Code, the CFR, and 50-state codes) is the slice we expose through our own statutes and regulations work and the public legal API; it exists precisely because "the model probably remembers the statute" is not a foundation you build legal work on.

The questions to actually ask in a demo

So drop "what are you trained on" from your evaluation script. It selects for the wrong thing. Here is what to ask instead, and what good answers sound like.

"What does it retrieve from at query time?" You want a named, real corpus of primary authority: federal and state opinions, the U.S. Code, the CFR, state codes. "Our model knows the law" is not an answer. "It searches these specific sources" is.

"Can I open every citation it gives me?" Every cite should link to the actual opinion or statute section. If you cannot click through to source, you cannot verify, and unverifiable is unusable for anything you file.

This is the single fastest hallucination test: ask a research question in the demo, then open all six citations. Dead links and "this case could not be located" are the tells.

"How does it handle a case that got overruled?" Throw it the Chevron question. A tool that confidently walks you through Chevron deference as current law in 2026 is answering from stale memory. A tool that surfaces Loper Bright and flags the overrule is doing the work.

"What is your hallucination rate, and how do you measure it?" The right answer is a number with a methodology, not "we don't hallucinate." Anyone claiming zero is either not testing or not telling you. The honest vendors will talk about retrieval grounding and the verification layer they put on top, because they know retrieval alone does not get to zero.

If you want the wider lay of the land on how the named players price and position around exactly this, we mapped it in the Harvey, Legora, and CoCounsel pricing reality, and the companion buyer question (where your data goes, not just where your answers come from) is covered in where your legal AI data actually goes.

Together they are the two questions that actually separate a tool you can build a practice on from one that looks great until the first citation check.

The one-line version

"Trained on federal and state case law" sounds like a strength and behaves like a confession.

Training teaches a model the shape of the law, not its text, and shape is exactly what fabricated citations are made of. The tools you can trust read from a real, current corpus of opinions and statutes, show you every source, and still ask you to verify.

Ask what a tool retrieves from and whether you can open the citations. The answer to that question tells you everything the training number was hiding.

FAQ

Is legal AI trained on case law? Some of it is, in the sense that large language models see legal text during training, but that is not what makes a tool reliable. The legal AI you can trust on citations retrieves real opinions and statutes at query time and grounds the answer in them. Training teaches the model the shape of legal language. Retrieval gives it the actual text to read and cite.

What is legal AI training data? It is the body of text a model learned from during training: web pages, books, and in some cases licensed legal corpora. A model does not store these as a searchable database. It adjusts its internal weights to predict plausible text, which is why a model answering from training memory can produce a citation that reads perfectly and does not exist.

Is training on case law better or worse than retrieving it? Worse for accuracy. A model answering from training memory can fabricate citations, because to the model a reporter volume and page number is just another sequence of tokens. A retrieval tool searches a real corpus first and answers only from what it finds, so every cite traces back to a source you can open.

Does Harvey or CoCounsel train on case law? Both describe their research as grounded in primary authority retrieved at query time with clickable citations, not as a model with case law baked into its weights. Thomson Reuters bought Casetext for 650 million dollars in 2023 to fold CoCounsel into a verified corpus and workflow, not to get a model that memorized cases.

Do retrieval-based legal AI tools still hallucinate? Yes, just less. The 2024 Stanford HAI study found purpose-built legal tools hallucinated at roughly 17% and 33%, against about 43% for a general GPT-4 baseline. Retrieval moves you from dangerous to usable. Your own verification of every citation moves you from usable to filable.

Why does a model trained before 2024 get current law wrong? Training has a cutoff date, and law moves fast. A model finalized before June 2024 still treats Chevron deference as good law even though the Supreme Court overruled it in Loper Bright v. Raimondo, 603 U.S. 369 (2024). Only a tool that retrieves the current opinion and checks how it was treated catches that.

What should I ask a legal AI vendor about training and accuracy? Drop "what are you trained on." Ask what the tool retrieves from at query time, whether you can open every citation it returns, how it handles an overruled case like Chevron, and what its measured hallucination rate is and how they tested it. Confident "we don't hallucinate" answers are the tell.

Try it on your next question

Vaquill AI is an AI legal research and drafting suite built for US firms, not AmLaw price tags. Ask a question and get an answer grounded in real federal and state court opinions, with citations you can open and verify. Compare documents, build chronologies, and keep every matter organized in one workspace.

Start free at app.vaquill.ai or see how Vaquill AI works.

Vaquill AI

Product & Content

Legal AI suite for US working lawyers: research, drafting, document comparison, document matrix, matters, and citation-verified answers, in one tool.