Evaluating a Legal Data Vendor: The Security and Deployment Questions to Ask

Ask a legal data vendor eight questions, in this order: what data of mine do you retain, do you train on it, where is it hosted, who is the contracting entity, who are your subprocessors, is there a signed DPA, how are credentials issued and revoked, and what happens to my data at contract end. Then ask about deployment. Most security reviews start at the certification badge and never get to the questions that decide whether an incident is survivable. This post is the checklist, written from the vendor side.

TL;DR

• Order matters. Retention, training, and hosting are the three questions that change your risk. Certifications are evidence about process, not answers about your data.
• A primary-law data API has a much smaller input surface than a document platform. You send search strings and citations, not your client's contracts. That shrinks question one dramatically, though it doesn't zero it out.
• A query is still customer input. "Cal. Civ. Code § 1798.150 class action exposure" leaks intent even though it contains no client document. Ask for the query-log retention window in writing.
• Ask for the report, not the badge. A certification logo tells you nothing about scope, period covered, or exceptions. Ask for the document behind it.
• Verify the coverage claim yourself before the security call. A free, machine-readable coverage endpoint turns a sales assertion into something your monitoring polls.

This post is part of our legal data infrastructure series.

For related coverage, see What a Legal Data API Should Guarantee: The Schema Contract, Legal Data Provenance: How to Tell Where a Statute Actually Came From, and Legal Data Freshness: What Refresh Cadence Actually Means, Source by Source.

First, figure out what actually crosses the boundary

Security reviews go wrong when the reviewer applies a document-platform threat model to a data API. The two have different shapes, and treating them the same wastes weeks on the wrong controls.

With a document platform, your client's material sits on the vendor's disk. With a primary-law data API, the corpus on the other side is public government text and the only thing you send is what you are looking for.

That is a real reduction in exposure, and any honest vendor should say so rather than sell you a bigger security story than the product needs.

Where the checklist came from

Vendor security reviews feel timeless. They aren't. The AICPA established its System and Organization Controls (SOC) reports in 2011, according to the Journal of Accountancy. A 2016 column in the same magazine explains the reason: businesses were outsourcing more and more to cloud providers and other service organizations, and they needed "valuable information" to "assess and address the risks associated with an outsourced service." Five years in, the columnist wrote, "I am seeing even more interest in SOC today than I have since it was introduced," because the move of business data to the cloud had people asking vendors for assurance. (Explaining the 3 faces of SOC, Journal of Accountancy, 2016.)

The report comes in two flavors that buyers mix up. The 2016 article explains that both SOC 1 and SOC 2 have Type 1 and Type 2 versions. A Type 1 report covers the design of controls "as of a specified date," while a Type 2 adds an opinion on whether the controls operated effectively "throughout a specified period" and describes the auditor's tests and results. The Journal of Accountancy's 2026 coverage adds that SOC 2 reports are "restricted to specified parties with sufficient knowledge and understanding of the service organization's system." So asking for the report is a real request, and a vendor may reasonably want a confidentiality agreement in place before it sends one.

It's also worth knowing that the badge itself is under some strain. In February 2026 the Journal of Accountancy reported that SOC technology vendors were flooding the market with promises of faster, cheaper reports, and that CPAs who do these examinations were worried. The chair of the AICPA's SOC 2 working group, Sean Linton, said SOC professionals "are seeing indications that 'fast and easy' may come at the expense of quality and objectivity." (Promises of 'fast and easy' threaten SOC credibility, Journal of Accountancy, February 2026.) None of that is a reason to distrust every report. It's a reason to read the one you're handed.

The eight questions

1. What data of mine do you retain?

Good answer: a specific list, by category, with a retention window for each. Query text, request metadata, billing records, and support tickets are four different things with four different lifetimes.

Bad answer: "we take security seriously" followed by an encryption paragraph. Encryption is about how data is stored. Retention is about whether it is stored at all, and it is the more important question.

Push until you get a number. "As long as necessary" isn't a retention policy, it is a way of not having one. Our own answers sit in the data privacy guide.

A stated deletion window is a default. Ask what happens to your data under a legal hold, meaning a court or regulator order to preserve records, and whether the contract obliges the vendor to tell you.

2. Do you train on it?

Good answer: a contractual commitment, in the DPA, that customer content and outputs aren't used to train or fine-tune any model, extended to the vendor's own AI subprocessors.

Bad answer: a toggle in a settings page. A setting can be flipped by a future product manager. A contract can't.

The follow-up that separates real from performative: does the commitment bind the vendor's model providers too, or only the vendor? A no-training promise that stops at the vendor's own boundary is worth much less than one that reaches the model API behind it.

3. Where is it hosted, and who is the contracting entity?

These are two questions and buyers routinely merge them.

Good answer: named cloud regions for data at rest and in process, plus the legal entity on the order form and where it is registered. A vendor should volunteer both without being pressed.

Bad answer: "the cloud" or "US-based" with no region named. Also bad: naming the region and going quiet about the entity, because your counterparty's domicile drives which courts and which regulators are in play if things go wrong.

4. Who are your subprocessors?

Good answer: a current register, available on request or published, plus written notice before a new one is added.

Bad answer: "we do not use subprocessors." Almost nobody is telling the truth when they say this. Cloud hosting, object storage, email, error monitoring, and payments are all subprocessors.

The useful test is whether the list includes the boring ones. A register with only the cloud provider on it hasn't been maintained.

5. Is there a signed DPA?

Good answer: a pre-executed DPA you can read before you talk to sales, aligned to the regimes that apply to you, with standard contractual clauses attached if any data crosses a border.

Bad answer: "we can look at your paper." That isn't a no, but it means the vendor has never done this, and you are about to spend six weeks in redlines on a data API that should take an afternoon to procure.

Read the retention and deletion clause first, then the subprocessor notice clause. Those two are where a readable DPA and a decorative one diverge.

6. How are API credentials issued and revoked?

Good answer: keys are created from a self-serve console, shown once, stored hashed, revocable instantly, and multiple keys per account are supported so you can rotate without downtime. A recognizable prefix helps secret scanners catch a leaked key in a public repo.

Bad answer: the vendor can email you a copy of your existing key. If they can show it to you, they are storing it in a recoverable form, and so is their backup system.

Ask what happens on revocation. A key that stops working within seconds is a control. A key that stops working at the next cache refresh is a hope.

Good key hygiene has a practical reason behind it. When GitHub redesigned its token formats in 2021, it explained that many of its old tokens were "hex-encoded 40 character strings that are indistinguishable from other encoded data like SHA hashes," which made detecting compromised ones "inefficient or even inaccurate." The fix was identifiable prefixes. (Behind GitHub's new authentication token formats, GitHub Blog, April 2021.) A visible prefix costs the vendor almost nothing, and it gives a scanner something to match when a key turns up in a public repo.

7. What happens at contract end?

Good answer: a stated export window, then deletion or anonymization, with the window written into the DPA rather than promised in a call.

Bad answer: silence, or a policy that lives only in a help article. Help articles change without notice and without a version history you can point at in a dispute.

For a data API there is a second half to this question that people forget. What happens to the data you already pulled? Check whether your license lets you keep and use cached primary-law text after the subscription ends, because the answer differs sharply between vendors who license their own editorial layer and vendors who serve government text.

8. What are my deployment options?

Good answer: an honest spectrum. Hosted API, hosted agent surface, and some path for buyers who can't take a runtime dependency at all.

Bad answer: "we can discuss on-premise" with no artifact behind it. Ask what exactly would be installed, who patches it, and whether anyone runs it today. An on-premise story that has never shipped is a sales answer.

What a complete answer set looks like

Any vendor worth buying from should be able to fill this in without a call. Use it as the response template, and treat a blank row as the finding.

Vaquill AI's own answers to every row sit on the security page, the DPA and the data privacy guide, which is where they belong: in documents you can attach to a security review, not in a blog post.

Raise these in procurement, not at signature

Two things are worth settling early with any data vendor, including us, because they are the ones that stall a security review at the last minute.

• Identity and network controls. Map how the credential will actually be used first. A key held in your backend's secret manager and used by one service is a different risk from a key handed to twelve analysts, and the control you need follows from that. Bring your requirement to the security page and the security review rather than assuming it from a feature grid.
• Service commitments. Uptime, support response, and remedies belong in the order form. Ask any vendor for the actual document rather than a stated percentage.

On certifications, read our security page for the current posture and then do the thing every experienced reviewer does anyway: ask for the report, not the badge. A logo on a website tells you nothing about scope, period covered, or exceptions. The report tells you all three.

What security reviewers say

The people who live inside these reviews are blunter than any vendor deck. A few of them, from Hacker News threads, with the caveat that they are individual opinions and not an audit standard.

On access, one commenter in a July 2026 SOC 2 thread made a practical point: "the SOC3 is a public report; the SOC2 report generally isn't supposed to be handed out except to named clients under contract." (tptacek, Hacker News, July 2026.) So ask which type of report you are looking at, what period it covers, and who is allowed to see it.

On exceptions, a commenter who said their customers are mostly large hospital systems described the stakes from the seller's side: "Anything except 'SOC2 compliant, no exceptions on report' will take an already long deal cycle (4-18 months) and double or triple it." (shinecantbeseen, Hacker News, July 2024.) Exceptions are the part of the report that tells you something, so read that section first.

And on the paperwork that surrounds all this, another commenter wrote: "We got a SOC2... and still get questionnaires." They described the questionnaires as the same ones sent to every vendor, with no follow-up questions afterward. (mrclark411, Hacker News, November 2021.) That's an argument for writing questions that can't be answered with a boilerplate yes: ask for a number, a document or a named entity, and treat a blank as the finding.

The deployment spectrum

Ask which shapes a vendor actually runs today, and ask who else is running them. A deployment option with no customer on it is a slide.

1. Hosted REST API. https://api.vaquill.ai/api/v1, bearer token, statutes routes at /us/statutes/*. Your code calls it, government text comes back with the publisher's own source URL attached. Key authentication and the per-plan rate limits are both documented. Lowest integration effort of the two.

2. Hosted MCP server. The same surface published at mcp.vaquill.ai on the same vq_key_ credential, so an agent can call it as a tool. Same surface, shaped for an agent rather than for your backend.

Why the open dataset matters to a security review

Every vendor in this category will tell you they are enterprise-ready. Very few will show you where the data came from.

The collection layer behind the corpus is published openly at github.com/Vaquill-AI/open-us-law under a permissive license, alongside the dataset itself. Site page at open-us-law, and every layer it runs, from a state website to an API response, is written up separately.

That changes what a security review can actually establish. Provenance stops being a claim in a questionnaire and becomes something your own engineers can read: which government source each corpus is built from, and how a document that fails a check gets refused rather than served.

One thing it doesn't change, because an unqualified claim here would be a lie.

Maintenance is the real cost of primary law. Government publishers change their HTML, move URLs, and rearrange bulk exports without notice. Keeping 56 jurisdictions current is continuous work, not a one-time import, which is most of what a buyer is actually paying for.

That is the whole argument for publishing the collection layer. A coverage claim you can audit beats a coverage claim in a sales deck, and it is the one form of trust a competitor can't copy by writing a better PDF.

Check the coverage claim before the security call

The one thing you can verify without talking to anyone:

curl -s https://api.vaquill.ai/api/v1/us/statutes/coverage \
  -H "Authorization: Bearer $VQ_KEY" \
  | jq '{measuredAt, totalSections}'

{
  "measuredAt": "2026-08-30T19:51:32Z",
  "totalSections": 4150839
}

The response also carries per-jurisdiction counts by corpus. measuredAt says when those counts were last measured, so you can diff them against yesterday's. Where a publisher states what a section is current through, that statement travels with the section as currentThrough. For when each source was last read, GET /boards returns cadence and lastRetrievedAt per source. Run these on a schedule and a coverage claim becomes something your monitoring checks rather than something a slide asserts. Neither call costs credits.

What to send your vendor

Paste the eight questions above into your security review as-is, alongside the schema questions that test response shape. Then add these three, which catch the most vendors off guard.

1. Show me the retention window for API query logs, in a document I can attach to the order form.
2. Name every subprocessor with production access, including error monitoring and support tooling.
3. If I cancel, what am I allowed to keep of the data I already retrieved?

If a vendor answers all three in writing within a week, they have done this before. If the answers arrive as a call invite, budget an extra month.

The standard worth holding this category to is simple: a buyer should be able to read the contract terms, the security posture, and the sourcing behind the data before speaking to anyone.

FAQ

What security questions should I ask a legal data API vendor?

Start with retention, training, hosting, contracting entity, subprocessors, DPA, credential issuance and revocation, and end-of-contract handling. Then ask about deployment options. Certifications are useful evidence about process, but they don't answer any of those eight questions on their own.

Is a legal data API lower risk than a legal document platform?

Usually yes, because the input surface is smaller. With a primary-law API you send search strings and citations rather than uploading client documents, so the vendor never holds your work product. The residual risk is that queries themselves reveal what you are working on, which is why query-log retention is worth asking about.

Does Vaquill AI train on customer data?

No. Customer content and outputs aren't used to train or fine-tune models, and the commitment sits in the Data Processing Agreement at /dpa rather than in a product setting. A contractual commitment is stronger than a toggle because a toggle can be changed by a future release.

How does the Vaquill AI API authenticate requests?

With API keys on a vq_key_ prefix, passed as a bearer token, and the same credential namespace serves the MCP server at mcp.vaquill.ai. Keys are created from the console, support rotation, and are revocable. Identity-management requirements beyond that belong in the security review, where they can be scoped to how your team will actually hold the credential.

Where do service commitments live in a legal data contract?

In the order form, not the marketing page. Uptime, support response, and remedies are negotiated terms, so ask any vendor for the actual document rather than a stated percentage, and raise the requirement early in procurement rather than at signature.

Where is Vaquill AI's data hosted?

In the regions recorded in our sub-processor register, which customers and their counsel can request. The DPA requires 30 days' written notice before primary data moves to another country. Data residency and the contracting entity are different questions; the operating entity is named in full on the DPA and the order form.

How can I verify where a legal data vendor's text actually came from?

Ask for provenance on the response itself: the publisher's own source URL on every result, plus a machine-readable coverage endpoint with counts and a measurement date, and the publisher's own currency statement on each section where one exists. Vaquill AI also publishes its collection layer and dataset openly at open-us-law, so the sourcing behind the corpus can be read rather than taken on trust.

How do I revoke a leaked API key?

Revoke it from the console and issue a new one. Look for a vendor that supports multiple live keys per account so you can rotate without downtime, shows a key exactly once at creation, and stores it hashed. If a vendor can email you your existing secret, that is a finding.

What should I check about a vendor's coverage claims before signing?

Ask for a machine-readable coverage endpoint you can poll. Vaquill AI's GET /us/statutes/coverage costs no credits and returns per-jurisdiction counts and a measuredAt timestamp, and each section carries currentThrough in the publisher's own words where the publisher states one. GET /boards adds cadence and lastRetrievedAt per source. A coverage claim your monitoring can check beats one you have to trust.

Priyansh Khodiyar

Co-Founder & CTO

Priyansh leads engineering and AI at Vaquill AI: the pipelines that pull statutes, regulations and court rules from every US jurisdiction's official publisher, and the REST API, MCP server and open dataset that serve them.