How to Run a Legal AI Pilot in 30 Days (2026)

To run a legal AI pilot in 30 days: pick one high-volume workflow (start with NDA triage), lock your data posture and baseline metrics in week zero, route real work through one tool for two to three weeks, then make a go or no-go call against a written scorecard in week four. The rest of this guide is the week-by-week version, with the metrics, the scorecard template, and the failure modes that sink most pilots.

The fastest way to lose a quarter is to start a legal AI pilot the way most teams do: spin up trials of four tools, let everyone "play with it," and reconvene in ninety days to compare gut feelings. You end up with anecdotes, not evidence, and a GC who still cannot tell the CFO whether the spend paid off.

A legal AI pilot run that way is not a test. It is a vibe check with a budget line.

There is a better version, and it fits in thirty days. The constraint is the point: thirty days forces you to pick one workflow, measure one baseline, and make one decision.

This is a playbook an in-house team can run between everything else on the queue, built to produce a number you can defend in a budget meeting.

Reviewed and corrected on 12 October 2026. This is general information, not legal advice. Third-party figures below are tied to the page we read on 12 October 2026, and the worked example is invented arithmetic, not data from a real pilot.

TL;DR

• Pilot one high-volume workflow (NDA triage or first-pass contract review), not the whole suite. Breadth is how pilots die.
• Write down your success metrics before day one: turnaround time, escape rate (errors that slip past the AI), and hours reclaimed. No baseline means no verdict, just impressions.
• Set data posture and access controls up front, in week zero, so the pilot itself is not the thing that leaks a privileged document.
• Run a tight week-by-week plan: week 1 setup and baseline, weeks 2 to 3 real work through the tool, week 4 measure and decide.
• The common failure modes are boringly consistent: no baseline, too many tools at once, and no named owner. MIT NANDA's July 2025 report says 95 percent of organizations are getting zero return from generative AI, on preliminary findings and a small sample. Treat it as a direction, not a precise failure rate. Its authors point to approach, not model quality.
• End on a go or no-go call against a written scorecard, not a meeting where the loudest opinion wins.

Where the method comes from

The structure draws on three things: a published study of why enterprise AI pilots stall (the MIT NANDA report below), the baseline-first ROI method that finance teams accept (Clio and Axiom both publish versions of it), and ordinary experimental design: change one variable, measure before and after, write the decision rule in advance. Third-party figures are named, dated and linked so you can check them.

Why most legal AI pilots fail before they start

The headline number is worth sitting with, with its caveats. MIT's Project NANDA, in its July 2025 report "The GenAI Divide: State of AI in Business 2025," says that despite 30 to 40 billion dollars in enterprise investment in generative AI, "95% of organizations are getting zero return," and that "just 5% of integrated AI pilots are extracting millions in value." The report labels itself preliminary findings. It rests on a review of more than 300 public AI initiatives, interviews with 52 organizations and a survey of 153 senior leaders (MIT NANDA report, PDF copy, read 12 October 2026).

That figure is cross-industry, not legal-specific, and the sample is small, so read it as a direction and not as a legal benchmark. The report's own diagnosis is that the divide "does not seem to be driven by model quality or regulation, but seems to be determined by approach," and it names the core barrier as learning: "Most GenAI systems do not retain feedback, adapt to context, or improve over time."

For a legal department, that gap has three usual shapes, all visible before you open the trial.

No baseline. The cardinal sin. If you do not know how long an NDA takes you today, "the AI feels fast" is not a finding, just a pile of warm impressions no finance partner signs off on.

Pilots rarely fail because the tool was bad. They fail because nobody measured the before, so there is no after to compare it to.

Too many tools at once. Running four trials in parallel feels rigorous and is the opposite. You split thin attention four ways, none of the tools gets enough real volume to show its true behavior, and you cannot isolate which tool produced which result. A pilot is an experiment. Change one variable.

No owner. When the pilot belongs to "the team," it belongs to no one. Someone has to own the baseline, route the real work through the tool, log the misses, and own the decision. Without a name attached, the pilot evaporates the first week the queue spikes, which is every week.

A fourth, subtler killer: misaligned incentives. If the pilot lawyers bear all the visible risk of a bad AI call but get no credit for time saved, the rational move is quiet non-compliance.

They run the pilot on easy contracts and do the hard ones the old way, so your pilot measures the AI on a sample it will never see in production.

There is a category error sitting underneath many of these failures. In-house teams often line up a point contract-review tool, the AI module in a contract lifecycle platform, a general assistant such as Microsoft Copilot, and a broader legal platform in the same evaluation, then grade them on one rubric, as if they were the same product.

They are built for different jobs, so a single pilot scored against a single workflow will flatter whichever tool happens to match that workflow. The fix is to be honest that you are testing one job, name the job, and accept that the winner of an NDA-triage pilot has earned the right to do NDA triage and nothing else yet.

Here are the four categories, with examples each vendor describes on its own site, and what to check in a pilot:

Score each against the one job you are testing, not against each other. A tool that loses an NDA-triage bake-off has not failed; it may never have been aimed at that job. For a vendor-by-vendor look at what each says it does in Word, see legal AI Word add-ins compared.

Week zero: pick the workflow and lock the data posture

Before the clock starts, do two things. They take an afternoon and decide whether the rest of the month means anything.

Pick one workflow. For most in-house teams the right pick is the thing clogging the queue: inbound NDA triage or first-pass review of a recurring contract type (vendor MSAs, order forms, DPAs). Choose by volume and repetition.

You want a workflow that runs often enough to generate real data inside three weeks and is standardized enough that "right" and "wrong" are knowable. NDAs are the canonical starter: you see dozens, the failure modes are well understood, and a miss is recoverable.

Do not pilot on your most bespoke, highest-stakes M&A paper. You will not get enough reps, and the variance will drown the signal.

Lock the data posture. The step in-house teams skip and regret. The pilot moves privileged, PII-laden documents into a vendor's system, and privilege does not protect itself once the file leaves your control.

This is not just a procurement formality; it sits inside your ethics duties. ABA Model Rule 1.6(c) says a lawyer "shall make reasonable efforts to prevent the inadvertent or unauthorized disclosure of, or unauthorized access to, information relating to the representation of a client." Comment [8] to Rule 1.1 tells lawyers to keep abreast of "the benefits and risks associated with relevant technology," and the LawSites tracker counts 40 states, the District of Columbia and Puerto Rico as having adopted that duty (read 12 October 2026). ABA Formal Opinion 512 (29 July 2024) applies these duties to generative AI; see our guide to Opinion 512. Check your own state's version of the rules.

Settle the posture before a single contract goes in, and get specifics in writing, not a "we take security seriously":

• Are prompts and outputs used to train the vendor's or model provider's models? You want a flat no in the contract, not the marketing page.
• Is there a zero-data-retention agreement with the model provider, and what is the vendor's own retention and deletion SLA?
• Is data US-resident and US-processed? Is a BAA available if any contracts touch PHI?
• SOC 2 Type II (the audited-over-time one, not a Type I snapshot), current sub-processor list, and notice before it changes.
• SSO, SCIM, role-based access, and audit logs, so you can prove later who saw which pilot matter.

Set access controls so only named pilot participants see pilot matters, segregated from your live matter store. Our vendor security questionnaire for in-house counsel is built for this gate; run the data, access, AI, and sub-processor sections before the trial, not at renewal.

Be precise about the legal exposure rather than reaching for "waiver" as a scare word: feeding privileged or work-product material to a vendor whose terms permit broad disclosure, training use, or open-ended retention can create confidentiality, attorney-client privilege, work-product, and professional-responsibility risk, and exactly how much depends on the controls and contract terms you put in place first.

Define success before you start: the three metrics that count

Pick your numbers now, while you have no result to rationalize. Three metrics carry almost all the weight for an in-house pilot, and they fit on one card.

Turnaround time. Median time from "contract lands in queue" to "reviewed and routed." Measure it as wall-clock, not active-work minutes, because the business feels calendar days, not your billing increments.

For a sanity range, Axiom's article on proving legal AI ROI says that in the use cases it tracks, AI-enabled workflows show "40 to 60 percent efficiency gains in areas like contract review and document analysis" (Axiom, read 12 October 2026). That is a vendor-side figure for its own use cases, not an audited benchmark and not a promise. Your number, measured on your paper, is the only one that counts.

Escape rate. The metric teams forget and the one that protects you: the share of AI-reviewed contracts where a material issue slipped past, caught only on human spot-check. Define "material" before you start so the count is not argued after the fact.

On an NDA pilot, material usually means a confidentiality obligation that came back non-mutual, an over-broad residuals clause, a weakened carve-out, an auto-renewal you would have struck, or governing law or venue that moved. On a vendor MSA, add liability cap erosion, indemnity expansion, and a missing DPA reference.

A tool that is fast and wrong is worse than slow and right, because it manufactures false confidence.

Have a senior reviewer re-check a fixed sample (every fifth contract is a sensible cadence) at full depth and log every miss against that list. If a vendor flinches at the escape-rate question, that is your answer.

Hours reclaimed. Lawyer hours returned per week on the piloted workflow. This is the number the CFO buys. Be conservative and net: time saved minus time spent supervising the AI and fixing its misses.

Vendors publish hours-saved figures, and some may be real for some teams, but treat every external figure as a claim until your own log confirms it on your own paper. As one example, GC AI's blog post says it interviewed more than 100 of its customers at the end of 2025 and reports an average of 14 hours saved per week and a 14 percent reduction in outside counsel spend. It adds that, applied to a $1.8 million median outside-counsel spend, 14 percent would be roughly $252,000 a year (GC AI, read 12 October 2026). Read that as a vendor's own customer survey. The pilot exists to produce your number, not import someone else's.

Scope and success metrics are set in week zero; the go/no-go decision is made on evidence in week four.

The week-by-week plan

Week 1: setup and baseline

Spend the first week proving the present, not testing the future. Pull your last thirty to sixty days of the chosen workflow and compute the baseline: median turnaround, current escape rate (yes, your human process has one too, you have just never measured it), and hours the workflow consumes per week.

Non-negotiable. The baseline is the only thing that turns week 4 into evidence instead of opinion.

In parallel, stand up the tool with the week-zero data posture in place. Load your actual templates and standards so the AI reviews against your positions, not generic ones (the MIT report's point about tools that never adapt to context applies here).

Encode your fallbacks and playbook now so the pilot tests the tool you would actually deploy. Pick the owner and the one or two lawyers who will run real work through it, and keep the circle small.

Weeks 2 to 3: real work, real volume

Route live contracts through the tool. Real inbound, not a curated test set. The lawyers still own the decision; the AI does the first pass. Every reviewer logs three things per contract: AI turnaround, whether they accepted, edited, or discarded the AI's read, and any miss the spot-check caught.

Two risks are worth planning for. First, reviewers may under-log edits. It is easy to quietly fix the AI's missed carve-out and move on, so the log shows "accepted" when the truth is "accepted after I rewrote it," and your hours-reclaimed number inflates. The fix is a dropdown, not a free-text box: accepted clean, accepted with edits, discarded, with edits forced into their own category.

Second, cherry-picking can creep in sideways. If reviewers choose what to feed the tool, the messy reseller paper and the heavily marked-up customer redlines can end up handled the old way while only the clean counter-signatures reach the tool. One sign is pilot volume that sags in exactly the weeks the queue spikes, the inverse of what real adoption looks like. Prevent it by pulling intake from the queue yourself rather than letting reviewers select what they feed the AI.

The single most useful thing a GC can ask in the week-four readout is not "is it good," it is "show me the three worst contracts it touched and what it missed on them," because the average case was never the question.

Resist adding a second tool "just to compare." You committed to one variable. If you genuinely need a head-to-head, run them sequentially on comparable volume, never tangled on the same week's queue. Two weeks of honest volume on one workflow tells you more than four tools dabbled with for a month.

Mind the sample size. Ten contracts is not a pilot, it is a coincidence; one good or bad run swings the whole number. This is the real reason to pick a high-volume workflow: an NDA stream that throws thirty to fifty contracts in three weeks gives you signal, while your quarterly enterprise MSA gives you three data points and a shrug.

The curated test set is the opposite trap: cherry-picked clean contracts make every tool look brilliant and tell you nothing about the messy paper in the real queue.

Week 4: measure and decide

Stop new intake mid-week and close the books. Compute the same three metrics you baselined and set them side by side. Turnaround: baseline versus pilot. Escape rate: human-only versus AI-assisted. Hours reclaimed: net, after supervision.

Then score it. Do not let the readout become a meeting where the most enthusiastic associate or the most skeptical partner sets the verdict by volume.

Make the math explicit, and be honest about what the sample can and cannot tell you. A hypothetical worked example, with invented round numbers (not data from any real pilot): you baseline 42 NDAs over the prior month at a median 2.5 days turnaround, roughly 21 lawyer-hours a week, with a human escape rate near 7 percent on full re-review.

In weeks 2 and 3 you route 38 live NDAs through the tool, median turnaround drops to 0.75 days, and your every-fifth-contract spot-check (8 of the 38, re-reviewed at full depth) surfaces one material miss. That is one miss in a sample of 8, a 12.5 percent rate in the sample, which is your best estimate of the true escape rate across all 38, not a measured fact about them.

With a sample that small the confidence band is wide: one more miss in those 8 would read as 25 percent, zero would read as 0, so do not treat the point estimate as precise. The honest read is "estimated escape rate roughly 12 percent, single-digit sample, low confidence." Net hours reclaimed land around 9 a week.

Now you have a real decision: turnaround crushed it, hours are material, but the estimated escape rate sits above your human baseline on thin evidence. Not an automatic yes, and not a clean no either.

It is a reason to fix the playbook and re-run the spot-check at a heavier sampling cadence before you scale, because the cheapest thing in a pilot is one more week of re-review, and the most expensive is scaling a 12 percent miss rate you never confirmed.

The pilot scorecard

Score each line 0 to 2 (0 fails, 1 marginal, 2 clears). This is the artifact that survives the meeting. Copy the table straight into the pilot memo.

The prose behind each line:

• Turnaround beat baseline by a margin you would pay for. Faster at all is not the bar. Pick the number before the pilot starts. Some teams start at a 25 percent median reduction; on a high-repetition workflow like NDAs, you may set it higher. (0-2)
• Escape rate at or below your human baseline. Score 2 only if the AI-assisted rate meets or beats your human number. More material issues slipping through means speed is a liability. (0-2)
• Net hours reclaimed are real and material. Time saved minus supervision and rework. Convert to dollars at a loaded rate and require it to clear a threshold you set in advance (a simple one: the annual seat cost, several times over). (0-2)
• Data posture cleared in writing. ZDR, no training on your data, residency, BAA where required. (0-2)
• Lawyers would keep using it unprompted. Adoption that needs nagging does not survive a busy week. (0-2)
• Total cost is known in writing and fits the budget. Get a written quote that covers the seats, add-ons and usage limits for the term you would actually sign. (0-2)

Add it up out of 12, and read the total against thresholds you wrote down before week 4, not after. A workable default scale: below 7 is a no-go, full stop, the tool did not earn the next workflow.

A 7 or 8 is a retune, not a verdict: you run exactly one fixed tuning cycle (tighten the playbook, reload templates, re-run a heavier spot-check sample) and re-score once, with no open-ended third try.

A 9 or higher is an expand, but only if the escape-rate line specifically cleared, because a high total carried by speed and adoption while material issues slip through is the most dangerous score on the card.

The go or no-go decision

A real decision is a threshold set in advance, not a feeling you arrive at. Set the bar before week 4 so you cannot move the goalposts to fit the result you want.

A clean go: the gate clears, the scorecard lands at 9 of 12 or higher, the escape-rate line cleared, and turnaround and net hours both beat baseline by a margin worth paying for. Expand to the next workflow; do not boil the ocean. Teams that scale AI well grow one proven workflow at a time.

A clean no-go: the gate fails, or the scorecard sits below 7 with marginal time savings and a worse escape rate. Not a wasted month. A pilot that produces a confident no with a documented reason is a successful pilot, and you now know what to demand from the next vendor.

The 7-to-8 band is its own answer, not a coin flip: run the single retune cycle, re-score, and let that second number decide. If it does not clear 9 after a focused tune, the tool is telling you it needs babysitting your team will not sustain.

The trap is the murky middle: decent numbers, no clear winner. Resist the urge to "extend the pilot to gather more data." Open-ended pilots are where good intentions go to die.

One honest exception: if your volume was genuinely too thin to read (a low-volume or seasonal department), a one-time, fixed extension to hit a real sample size is legitimate, as long as you set the new end date and threshold up front.

Absent that, if thirty days of real volume on your highest-frequency workflow did not produce a clear signal, the signal is weak, and weak signal on the easy workflow is a no-go on the hard ones. Decide, write down why, and move.

What this approach buys you

The discipline is the deliverable. A thirty-day pilot run this way produces three things a sprawling ninety-day trial rarely does: a defensible number for the budget conversation, a documented data posture you can point to if anyone challenges the privilege question, and a team that has used the tool on real work rather than poked at a demo.

Whichever way it lands, you come out with evidence instead of opinions, which is the whole point of a pilot rather than a launch.

If the workflow you pick is contract review, our in-house contract review playbook gives you the escalation tiers and fallback positions to baseline against, and the complete guide to legal AI for in-house counsel covers the broader buying criteria a pilot confirms. Once you have a go, the guide to rolling out legal AI to your team takes the proven workflow into wider use, the legal department KPIs to measure keep the gains visible past month one, and how AI is transforming in-house legal teams sets the wider context a single pilot sits inside.

FAQ

How long should a legal AI pilot last?

Thirty days of real volume on one workflow is enough for most in-house teams, and the constraint is a feature: it forces a single workflow, a single baseline, and a single decision. Outside guides land in a wider band (Axiom's article describes a structured 8 to 12 week pilot to turn efficiency gains into visible budget impact), but that length is driven by tool count and scope. Narrow both and a month produces a defensible number. Extend only once, with a fixed new end date, and only if your volume was genuinely too thin to read.

What use case should an AI pilot for a legal team start with?

Start with the highest-volume, most standardized thing in your queue, which for most teams is inbound NDA triage or first-pass review of a recurring contract type such as vendor MSAs, order forms, or DPAs. You want enough reps to generate signal inside three weeks and a workflow where "right" and "wrong" are knowable. Do not pilot on bespoke, high-stakes M&A paper; the variance drowns the signal.

How do you measure the ROI of a legal AI pilot?

Baseline first, then compare three metrics: turnaround time, escape rate (material issues that slip past the AI), and net hours reclaimed after supervision and rework. Convert the hours to dollars at a loaded rate and set the dollar threshold before week four. Clio and Axiom both publish baseline-first versions of this method: Clio's five-step framework starts by identifying the workflow and setting pre-AI baseline metrics, and Axiom says baseline data should be established before deployment (Clio; Axiom).

How many AI tools should you pilot at once?

One. Running four trials in parallel splits your attention, denies each tool enough real volume to show its true behavior, and makes it impossible to isolate which tool produced which result. A pilot is an experiment, so change one variable. If you genuinely need a head-to-head, run the tools sequentially on comparable volume, never tangled on the same week's queue.

What metrics prove a legal AI pilot worked?

Three carry almost all the weight: median turnaround time from queue to routed, escape rate against your own human baseline, and net hours reclaimed per week. Speed alone is a trap, because a tool that is fast and wrong manufactures false confidence. Score escape rate as a gate, not a tiebreaker.

Why do most legal AI pilots fail?

The model is rarely the problem. Pilots fail on process: no baseline to compare against, too many tools at once, no named owner, and quiet cherry-picking where lawyers run the AI on easy contracts and do the hard ones the old way. MIT NANDA's July 2025 report says 95 percent of organizations are getting zero return from generative AI (preliminary findings on a small sample) and attributes the divide to approach, with the core barrier being that most systems do not retain feedback or adapt to context (MIT NANDA report).

Who should own a legal AI pilot?

One named person, not "the team." Someone has to own the baseline, route real work through the tool, log the misses, and own the go or no-go call. Without a name attached, the pilot evaporates the first week the queue spikes, which is every week.

Can you run a legal AI pilot in one week?

You can run a one-week smoke test, and many vendor trials are short. It answers "does this tool basically work on my paper," which is worth knowing before you commit a month. What it cannot give you is a defensible number: one week rarely clears thirty contracts on a single workflow, and an escape rate estimated on a handful of spot-checks has a confidence band too wide to scale on. Use the week to disqualify obvious misses, then run the thirty days on the survivor.

How do you stop lawyers using unsanctioned AI during the pilot?

You mostly do not stop it by policy; you displace it by giving people a sanctioned tool that is faster than the consumer chatbot they are already pasting into. That is a real reason to sanction one vetted option early rather than leave the team in a no-approved-tool limbo, where the pressured move is shadow AI with no data posture behind it. Track whether unsanctioned use drops during the pilot as a secondary signal.

For the US statutes, regulations and court rules these laws sit in, see the US primary law API.

Arshita Anand

Co-Founder & CEO · Attorney

Arshita leads product and strategy at Vaquill AI, the US primary law data API. As an attorney, she decides which law we carry, how each section is sourced and cited, and what legal AI and compliance teams need from the data.