Skip to main content

Domain-Specific vs General-Purpose Legal AI: When Specialized Wins

Short answer: A general-purpose model is a reasonable choice when the work is low-stakes and you will read the output yourself: brainstorming, summarizing a document you already have, a rough first draft. Move to a tool built for legal work when the output carries a citation, touches a live matter or will be read by a court, a regulator or a board. The published research supports a narrow version of the reason. The studies suggest general models make things up less often when they work from text you hand them than when they have to recall specific law (the tests differ, so the numbers are not comparable), but they still make things up in both cases, and the legal tools built to fix that have been measured making errors too. The buying decision is therefore about the task and about evidence, not about the label on the product. This is general information, not legal advice.

Reviewed and corrected on 12 October 2026. Figures below come from the sources named next to them, which we read on that date. Benchmarks and products change quickly, so check the sources for the current numbers.

TL;DR

  • The split is about the task, not the vendor. Summarizing text you supply and recalling specific law are different jobs, and the published error rates differ by job. Match the tool to the job.
  • General models are weak at recalling case law. In Stanford's study of 2023 models, published in the Journal of Legal Analysis in June 2024, models hallucinated between 58 and 88 percent of the time on a direct, verifiable question about a randomly selected federal court case.
  • Summarizing is not error-free either. Vectara's public leaderboard, updated September 22, 2026, puts hallucination rates for summarizing documents between 1.8 and 24.2 percent across the models it lists, with a median near 9.6 percent.
  • Purpose-built legal tools reduce the problem and do not remove it. A preregistered Stanford evaluation published in 2025 found that leading commercial AI legal research tools hallucinated between 17 and 33 percent of the time, less than a general chatbot but not zero.
  • Analyst forecasts are about models, not about any legal product. Gartner predicts that by 2028 more than half of the GenAI models enterprises use will be domain-specific. Treat it as a forecast.
  • Ask any vendor how it measured. A claim of fewer hallucinations means little without the task, the baseline and who ran the test.
Spot the problem
Question 1 of 4

Your team uses a general chatbot to summarize a 40-page vendor contract you uploaded, and also to "give me the leading case" on a clause. Which task is riskier, and why?

What "domain-specific" means

A general-purpose model is one system trained to do many things: write a poem, debug code, answer a medical question, draft a contract clause. GPT, Claude and Gemini are the usual examples.

"Domain-specific" gets used for two different things. Gartner defines domain-specific language models as models "trained or fine-tuned on specialized data for a particular industry, function, or process." In legal marketing it can also mean a product wrapped around a model, with retrieval from legal sources, citation checks and workflow controls. This post is mostly about the second meaning, because that is what a buyer is choosing between. Keep the two apart when you read a forecast or a vendor claim.

Some general-purpose vendors now ship legal-specific features too. Perplexity, for example, announced Computer for Counsel on June 24, 2026, with connectors to legal research sources, according to Pulse 2. We cover that wave in general-purpose AI enters legal. A legal mode can be a good starting point. What it does for accuracy is a question for evidence, which is the next section.

What the published research shows

Three public sources give a rough picture. They measure different things, so do not compare the numbers directly.

Question testedWho and whenResult
Summarizing a document, using only the facts in itVectara Hallucination Leaderboard, updated September 22, 2026Hallucination rates from 1.8 percent to 24.2 percent across the listed models; median about 9.6 percent
A direct, verifiable question about a randomly selected federal court case (2023 models)Dahl, Magesh, Suzgun and Ho, Journal of Legal Analysis, published June 26, 202458 to 88 percent
Commercial AI legal research tools on a preregistered set of legal queriesMagesh and others, Journal of Empirical Legal Studies, accepted March 14, 2025Each of the tools tested hallucinated between 17 and 33 percent of the time, fewer than a general chatbot

Summarizing. Vectara's open leaderboard uses its own evaluation model to score how often an LLM introduces claims that are not in the document it was asked to summarize. In the September 22, 2026 update, the lowest rate in the table is 1.8 percent and the highest is 24.2 percent. More than a dozen models sit at 15 percent or more. Summarizing from supplied text still leaves a real error rate, which is why you read summaries against the source.

Recalling case law. Stanford RegLab's "Large Legal Fictions" tested several general-purpose models. The authors report hallucination between 58 and 88 percent of the time when asked a direct, verifiable question about a randomly selected federal court case, and rates of at least 63 percent when asked for a case's central holding. These were 2023 models, so the figures show a pattern and are not a measure of today's models. An early preprint of the work, announced in January 2024, reported 69 to 88 percent and at least 75 percent on holdings. The published version reports the numbers above.

Specialized tools. Stanford's "Hallucination-Free?" study (Journal of Empirical Legal Studies, 2025) tested commercial legal research tools. It says that while hallucinations are reduced relative to a general-purpose chatbot, the tools each hallucinate between 17 and 33 percent of the time, and that the providers' claims of "eliminating" hallucinations or being "hallucination-free" are overstated. The authors also found substantial differences between systems. That study reflects the products as tested, and products change.

Why the design of the tool still matters

A general chatbot answers a legal question by generating plausible text. If it does not know the citation, it can produce one that looks right. We walk through how that gets lawyers into trouble in AI hallucination sanctions tracker.

A tool built for legal work is designed to change the mechanics: retrieve from a stated set of legal sources first, answer from what was retrieved, and make each citation checkable. That moves part of the job from recall toward reading, which the data above suggests helps. The Stanford study of commercial tools shows it does not eliminate the problem, so a design claim has to be matched by evidence that you have seen.

Loading diagram...

The decision framework: when general is enough, when it is not

You do not need a specialized tool for every task. Here is a working line.

A general model is usually reasonable when:

  • You are brainstorming, outlining or pressure-testing an argument you will verify yourself.
  • You are summarizing a document you already have, and you will check the summary against it.
  • The draft is a throwaway first pass that a human will rewrite before anyone sees it.
  • The stakes of an error are low and self-contained.

You want a stronger layer of sourcing and checking when:

  • The output carries a citation to a case, statute or regulation. Someone will check it, or worse, will not.
  • The work is matter-bound: it touches a specific client or deal.
  • It is compliance or regulatory work where a wrong standard is a liability event.
  • Anything a court, regulator or board reads under your name.

What the ABA says about how much to check

Under ABA Formal Opinion 512, the verification a lawyer owes "will necessarily depend on the GAI tool and the specific task that it performs." The opinion adds that a tool "designed specifically for the practice of law or to perform a discrete legal task" may call for less checking "where a lawyer's prior experience with the GAI tool provides a reasonable basis for relying on its results," and that lawyers "may not abdicate their responsibilities by relying solely on a GAI tool" for tasks that need professional judgment. It also cites the Stanford study of legal research tools as finding that leading providers' systems "hallucinate between 17% and 33% of the time."

The practical point: a tool built for the law can justify less checking only after you have experience with it. That experience has to come from testing it.

Test it yourself

The most reliable evidence is your own. Before you commit, run a short bake-off.

  1. Collect 20 to 30 questions from your own work where you already know the right answer and the controlling authority.
  2. Run the same questions through each candidate tool, general and specialized.
  3. For every answer, open each cited authority and check three things: that it exists, that it says what the answer says, and that it is current and applies in your jurisdiction.
  4. Record the misses by type: fake cite, wrong holding, wrong jurisdiction, out of date.
  5. Repeat on a harder set a month later, because models and products change.

Opinion 512's own example is the same idea: a lawyer who has tested a tool's accuracy on a smaller subset of documents has a basis for relying on it. Your bake-off gives you numbers you can defend.

The hybrid pattern and what the forecasts say

Some teams end up with both: a general model for reading, drafting and rephrasing, and a source-grounded tool for anything that cites law. That is a common pattern, not a proven winner, and we have no study that says it outperforms either alone.

Analyst forecasts point toward specialization in general. Gartner's April 9, 2025 press release predicts that by 2027, organizations will implement small, task-specific AI models with usage volume at least three times that of general-purpose large language models. Its October 20, 2025 release on 2026 strategic technology trends says "By 2028, Gartner predicts that over half of the GenAI models used by enterprises will be domain-specific." Both are predictions about models, and Gartner's definition of domain-specific models is about training or fine-tuning on specialized data. Neither is a measurement of how accurate any legal product is.

For how leading suites compare, see which legal AI is best for agentic suites, and for the in-house buying angle, legal AI for in-house counsel.

The verdict

Use a general-purpose model for reading, drafting and brainstorming where you check the result. Use a tool built around sources you can open for anything that cites law, and test it on your own questions before you trust it, because the best published evidence says even those tools get things wrong. The decision rule is the same either way: the more it costs if the output is wrong, the more of it a human has to read against the primary source.

For the US statutes, regulations and court rules these laws sit in, see the US primary law API.

Sources, all read on 12 October 2026: Vectara, Hallucination Leaderboard (last updated September 22, 2026). Dahl, Magesh, Suzgun and Ho, Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models, Journal of Legal Analysis 16(1), 64 to 93 (published June 26, 2024), and the Stanford HAI announcement of the preprint, Hallucinating Law (January 11, 2024). Magesh and others, Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools, Journal of Empirical Legal Studies (2025). Gartner, press release of April 9, 2025 and press release of October 20, 2025. American Bar Association, Formal Opinion 512 (July 29, 2024).

FAQ

What is the difference between domain-specific and general-purpose AI? A general-purpose model is trained to handle many tasks, from coding to poetry to legal drafting. "Domain-specific" is used for models trained or fine-tuned on specialized data and also for products that wrap a model in retrieval, checking and controls for one field. Ask a vendor which it means.

Are domain-specific legal tools more accurate than general models? Stanford's preregistered 2025 study found leading commercial legal research tools hallucinated less than a general chatbot, at between 17 and 33 percent. It also found the providers' claims of eliminating hallucinations were overstated. So they can help, and they still need checking.

When is a general-purpose model good enough for legal work? When the output is low-stakes and you will verify it yourself: brainstorming, outlining or summarizing a document you already have. Summaries still need checking against the source, because even on summarization benchmarks models have measurable error rates.

Do domain-specific tools still hallucinate? Yes, according to the Stanford study of commercial legal research tools. The aim of a well-designed tool is to make errors easier to catch, for example by linking each citation to a source you can open. It does not remove your duty to read the authority.

Is a hybrid approach better than picking one? Some teams use a general model for reading and drafting and a source-grounded tool for cited work. We found no study showing the hybrid beats either alone, so test it on your own questions.

Will general-purpose AI make specialized legal tools unnecessary? We do not know. Some general-purpose vendors are adding legal connectors and features, and the research above shows both kinds of tool still make errors. Judge each by its tested accuracy on your work.

How do I evaluate a legal AI vendor? Ask where the legal sources come from, how citations are checked before they reach you, how matters are kept separate, and how any accuracy figure was measured, on what task, against what baseline and by whom. Then run your own test on 20 to 30 questions you can already answer.

Connect our US primary law database.
Every US statute, regulation, constitution, and executive order via REST, MCP or SQL. 6M+ sections, section-level citations, and links to the official source. Plus a free open dataset.
Updated October 12, 202616 min read

New legal AI guides, weekly.

Arshita Anand

Arshita Anand

Co-Founder & CEO · Attorney

Arshita leads product and strategy at Vaquill AI, the US primary law data API. As an attorney, she decides which law we carry, how each section is sourced and cited, and what legal AI and compliance teams need from the data.

Cookie preferences

Choose what Vaquill may use on this browser. You can change this at any time from “Cookie settings” in the footer.

Details in the Cookie Policy.