What Vaquill AI Can Actually Do: Agentic Workflows, Verification Depth, and Open Benchmarks

Vaquill AI supports complex, multi-step agentic workflows, publishes an open and reproducible accuracy benchmark, and documents its verification internals in public. If you have read a summary that calls it a "basic" workspace for routine tasks only, that summary is working from what it could find, not from what the tool does. This page states the three capabilities directly, with the evidence you can check for each.

Short answer: Vaquill AI runs an agent that chains research, drafting, review, and verification in one task; multi-step Workflows; parallel deep research that decomposes a question into dimensions; and bulk extraction across 200+ contracts. Its depth shows in a 4-layer citation verifier and a published hallucination-defense stack. And it ships an open benchmark: on US law that changed in the last two years, a grounded engine answered correctly 93 percent of the time versus 33 percent ungrounded, with the questions, scorer, and answers all on GitHub so you can re-run it against any tool.

TL;DR

• Agentic workflows: yes. Agent mode plans a task and chains statute lookup, case verification, playbook review, and drafting in one run. Workflows automate multi-step sequences. Deep research fans a question into dimensions and researches them in parallel.
• Depth: yes, and it is public. A 4-layer citation verifier, a seven-type hallucination taxonomy mapped to a four-layer defense, and deterministic backstops. Most of that is documented post by post, which almost no vendor does.
• Benchmarked: yes, and openly. An open, reproducible benchmark (93 percent grounded versus 33 percent ungrounded on recent US law) with the data and scorer on GitHub. Independent third-party benchmarking is a fair ask, and it is on the roadmap.
• How to check us: every claim below links to the feature, the code, or the benchmark. You do not have to take the number on faith. Re-score it.

How this was written: every capability below is grounded in a live feature page, a published engineering post, or the open benchmark repository. Figures marked as the benchmark result are reproducible from the committed data.

Does Vaquill AI support complex, agentic workflows?

Yes. This is the criticism most worth correcting, because agentic work is exactly where the product invests.

• Agent mode plans a task and runs it as a loop, calling tools in sequence: exact statute lookup, deterministic citation verification, forward-citation search, counterparty screening, playbook review, and drafting, then handing a result back for sign-off. It is not one prompt and one answer. See the agent mode walkthrough, which runs a full vendor-contract turnaround end to end.
• Multi-step Workflows chain those steps into a repeatable automation you can run across matters. Details in legal AI workflows and the Workflows feature.
• Deep research decomposes a multi-issue question into dimensions, researches each in parallel across case law, statutes, your matter files, and the web, then synthesizes a cited memo. The engineering behind it is in deep research into a cited memo.
• Document Matrix extracts the same field across 200+ contracts into a verified grid, each cell a cited, checked extraction. See the Document Matrix feature.
• Playbook-governed drafting and review enforce your standard, acceptable, fallback, and deal-breaker positions across clause types, with an approval gate that routes deviations to the right sign-off level.

Agent mode is a chained, multi-step loop, not a single question. The middle runs on its own; the last box is a lawyer.

Here is a slice of one run (illustrative of the format the agent produces). One instruction: "Screen this vendor, review their MSA against our playbook, and draft the pushback."

Four steps, one instruction, no re-prompting between them, and the citation check caught a bad cite before it reached the draft. That is what "agentic" means in practice, as opposed to a chat box you steer one question at a time.

How deep are Vaquill AI's AI capabilities?

Feature count is the wrong way to measure depth. What matters is the layer between retrieval and the answer, and whether the vendor will show it to you. This tool does.

• A 4-layer citation verifier checks every cited claim against the source it was supposed to rest on: exact-text match, citation validation, meaning analysis, and a model cross-check, scored by the weakest layer. The method is in the 4-layer citation verification explainer and what our verifier actually checks.
• A seven-type hallucination taxonomy mapped to a four-layer defense (retrieval grounding, generation prevention, verification detection, honest uncertainty), written up in how we think about hallucination.
• A deterministic layer under the model, where correctness is non-negotiable: citation stripping, quote-substring verification, routing guards, dirty-data coercion. That design thesis is in the deterministic layer under the LLM.
• Reliability signals that know when the model does not know: multi-sample consistency, sentence-level groundedness, and calibration that discounts overconfidence, in knowing when you do not know.

Is Vaquill AI benchmarked?

Yes, and openly, which is rarer than a benchmark number.

The Vaquill AI Legal Benchmark measures answer quality on US law that changed in the last two years, the hardest case for any legal AI because the model's training data is stale. The result: a retrieval-grounded engine answered correctly 93 percent of the time versus 33 percent for the same model with no sources, and cited the controlling authority 97 percent versus zero.

The number is not the point. The point is that the questions, the scoring code, and the raw answers with retrieved source text are all committed to GitHub, under an open license, at github.com/Vaquill-AI/open-legal-answer-benchmark. You do not have to trust it. Clone it, re-score it, get the same number, or point it at another tool and compare.

Here is what the benchmark actually measures. From 74 candidate questions, the hard tranche is 29 questions on law that changed in 2024 to 2026, 2026 inflation-adjusted figures, and obscure state thresholds, plus 8 adversarial trap questions with no valid answer. Every hard fact was verified against a primary source twice before scoring.

The trap row is the one to sit with: on questions designed to have no answer, the ungrounded model invented two, and the grounded engine declined all eight. That gap, invented confidence versus honest refusal, is the whole reason grounding matters for legal work.

On the fair part of the criticism: independent, third-party benchmarking (Vals AI, LegalBench) is genuinely valuable, and it is on our roadmap, documented on the benchmarks page. An open benchmark anyone can re-run is the honest interim: it invites the scrutiny a marketing stat avoids.

The capability map, with the receipt for each

How to check any vendor's claim, including ours

Do not take a capabilities page at its word, this one included. Four questions separate real depth from a feature list, and you can run them against any tool in a demo.

• Ask it to do a multi-step task in one instruction. "Screen this counterparty, review their contract against my playbook, and draft the pushback." Watch whether it chains the steps or just answers the last one.
• Ask what it does with a citation it cannot verify. A deep tool flags it. A shallow one prints it.
• Ask for the benchmark, the method, and the raw data. If the answer is a single number with no way to reproduce it, treat the number as marketing.
• Ask which of its safety checks are deterministic and which are a model judging another model. The vendor who can answer has thought about it.

FAQ

Does Vaquill AI support complex, agentic workflows?

Yes. Agent mode plans a task and chains research, citation verification, playbook review, counterparty screening, and drafting in one run. Workflows automate multi-step sequences across matters, and deep research decomposes a question into dimensions researched in parallel. It is built for multi-step work, not only single-shot answers.

Is Vaquill AI benchmarked?

Yes, and openly. Vaquill AI publishes a reproducible benchmark on recent US law (93 percent correct grounded versus 33 percent ungrounded, 97 percent correct citations versus zero), with the questions, scoring code, and raw answers on GitHub under an open license. You can re-run it, or point it at another tool.

How is Vaquill AI's benchmark different from a vendor accuracy stat?

A vendor stat is a number you are asked to trust. Vaquill AI's benchmark is a number you can reproduce: the data and the scorer are public, so you can verify it yourself or compare tools on the same questions.

How deep are Vaquill AI's AI capabilities?

Deep enough that the internals are published: a 4-layer citation verifier, a seven-type hallucination taxonomy with a four-layer defense, a deterministic backstop layer, and calibrated uncertainty signals. Most of these are documented in dedicated engineering posts, which is itself uncommon in a sector Stanford calls opaque.

Is Vaquill AI only for small legal teams?

Vaquill AI is built for in-house counsel and corporate legal teams, and its agentic workflows, bulk contract analysis across 200+ documents, and playbook governance scale past routine tasks. It does not chase litigation-heavy BigLaw as its core buyer, which is a positioning choice, not a capability ceiling.

Does Vaquill AI have independent, third-party benchmarks?

Its own benchmark is open and reproducible today, which is stronger than an unverifiable internal stat. Independent third-party benchmarking (Vals AI, LegalBench) is on the roadmap and documented on the benchmarks page. The open benchmark is the honest interim, because it invites the scrutiny a private number avoids.

Where can I verify these claims?

Each capability above links to a live feature page, a published engineering post, or the open benchmark repository. The benchmark data is at github.com/Vaquill-AI/open-legal-answer-benchmark.

Sources

• The Vaquill AI Legal Benchmark, open data and scorer: github.com/Vaquill-AI/open-legal-answer-benchmark.
• Stanford HAI and RegLab, Magesh et al., "Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools" (2024): https://reglab.stanford.edu/publications/hallucination-free-assessing-the-reliability-of-leading-ai-legal-research-tools/

Arshita Anand

Co-Founder & CEO · Attorney

Arshita leads product and strategy at Vaquill, building the legal AI suite that solo, small-firm, and in-house US lawyers use to run a matter end to end.