Legal AI for Healthcare In-House Counsel: HIPAA, BAAs, and Vendor Risk

Part of the complete guide to Legal AI for In-House Counsel.

A deputy GC at a regional health system told me about the deal she killed last quarter. A well-funded legal AI vendor on her shortlist had answered every security questionnaire cleanly. SOC 2 Type II, ZDR claim with the model provider, US-only hosting, the usual checklist.

Then she asked one question: "Will you sign our BAA?" The answer was a paragraph long, ended in "we're working on a BAA program," and pointed her at a webinar. She moved to the next vendor that afternoon.

That moment, the BAA gate, separates healthcare in-house procurement from every other in-house legal AI buyer's problem. Most legal AI sales cycles end at the DPA. Healthcare's begin there.

The Business Associate Agreement is not a procurement artifact in this segment. It is the product boundary.

Short answer: Legal AI for healthcare is any drafting, research, or review tool that can lawfully touch protected health information (PHI). That means three things together: a signed Business Associate Agreement (BAA), a BAA chain that reaches the underlying LLM provider, and HIPAA-grade infrastructure. A tool that has all three can see PHI. A tool that has only SOC 2 and a DPA cannot, no matter how good the demo looks. Consumer ChatGPT, Claude, and Gemini never qualify, because no consumer tier signs a BAA.

TL;DR

Part of our in-house counsel guide series.

• Healthcare in-house counsel face a harder legal AI vendor problem than peers in fintech or SaaS. HIPAA plus state overlays (NY SHIELD, Cal. CMIA, Tex. HB 300, Washington MHMDA) eliminate most general-purpose legal AI tools before the demo.
• Only vendors with an executed BAA, a full downstream BAA chain to the LLM provider, and verifiable HIPAA-grade infrastructure should touch PHI. "Don't put PHI in the prompt" is not a workable policy at scale and the auditors know it.
• OCR's Risk Analysis Initiative, announced in late 2024, has produced a sequence of publicly reported BA enforcement actions tied to ransomware. The widely cited BST & Co. CPAs resolution announced in August 2025 ($175,000 plus a two-year CAP) is the template.
• Six structural vendor categories meet the bar in 2026. General-purpose legal AI passes only at the enterprise tier; consumer LLMs never pass.

Why healthcare in-house faces a sharper problem

Every in-house team buying legal AI runs a vendor diligence motion. Most stop at SOC 2, a signed DPA, and a confidentiality flow-down.

Healthcare cannot stop there. HIPAA sets a federal floor that, in practice, eliminates most of the legal AI vendor market before the demo even starts.

The trigger is 45 C.F.R. § 160.103, which defines a "business associate" as a person or entity that creates, receives, maintains, or transmits PHI on behalf of a covered entity. A legal AI tool that routinely ingests memos, deposition prep, or contract redlines containing identifiable patient information, on behalf of a covered entity, is functioning as a business associate.

The analysis turns on the actual data flow, not the lawyer's intent. A tool used only on de-identified or synthetic data may fall outside; a tool routinely ingesting real records does not.

The Privacy Rule, 45 C.F.R. § 164.502(e), then prohibits a covered entity from disclosing PHI to a business associate without a written agreement that satisfies § 164.504(e). The Security Rule, 45 C.F.R. § 164.314(a), layers on the contractual obligation that the BA implement administrative, physical, and technical safeguards under §§ 164.308, 164.310, and 164.312.

A real BAA is not a one-page addendum. It commits the vendor to breach notification under 45 C.F.R. § 164.410, to subcontractor BAAs all the way down the stack, to permitted-use language that does not match a typical SaaS T&C, and to a documented risk analysis under § 164.308(a)(1)(ii)(A).

The vendor's underlying model provider, whether OpenAI, Anthropic, or an Azure OpenAI deployment, needs to be inside that BAA chain. If the vendor cannot show a BAA with the downstream LLM provider for the specific deployment, your covered entity cannot lawfully send PHI through that pipeline.

OCR has been making this point loudly. The Risk Analysis Initiative, announced in late 2024, has produced a sequence of publicly reported enforcement actions against business associates, most tied to ransomware.

The BST & Co. CPAs resolution, announced in August 2025, resolved a December 2019 ransomware incident for $175,000 plus a two-year corrective action plan, with OCR's resolution materials pointing to a risk analysis that did not sufficiently cover all systems and locations where ePHI was present.

Other 2025 BA settlements landed in the mid five figures to low six figures with the same fact pattern: cloud-hosting or EHR-adjacent vendor, ransomware, thin risk analysis. Every one of those BAs answered a security questionnaire before the breach.

The missing-BAA failure mode is older and just as expensive. In December 2018, OCR settled with Advanced Care Hospitalists, a Florida physician group, for $500,000 plus a corrective action plan after it disclosed the PHI of 9,255 patients to a billing contractor with no BAA in place (OCR resolution, December 2018; reported by HIPAA Journal). The lesson maps directly onto AI procurement: a tool that touches PHI without a signed BAA is not a low-rated vendor, it is an unauthorized disclosure the moment data flows.

HHS has separately signaled, in its January 2025 Notice of Proposed Rulemaking updating the Security Rule, that entities using AI tools should include those tools inside their risk analysis and risk management activities.

Whether or not the final rule lands in the proposed form, the posture is settled: your legal AI vendor is a named, documented entry in your risk analysis. Not a footnote.

The de-identification escape hatch (and why it rarely helps)

The one clean way to use a non-BAA AI tool on health data is to take the data out of HIPAA's scope first. HIPAA stops applying to information that has been de-identified under the Privacy Rule, 45 C.F.R. § 164.514. There are two paths: Safe Harbor and Expert Determination.

Safe Harbor requires stripping all 18 identifier categories listed in § 164.514(b)(2), including names, geographic subdivisions smaller than a state, all date elements more specific than a year, contact details, account and record numbers, biometrics, and full-face photos. Remove all 18, with no actual knowledge that the remainder could re-identify someone, and the data is no longer PHI.

Expert Determination lets a qualified statistician certify that the re-identification risk is very small. It is the path used for richer datasets where Safe Harbor would gut the data's usefulness.

Here is why this rarely rescues a legal AI workflow. In-house counsel work on the identified record by definition. A privileged memo about one patient's adverse event, a deposition prep file, a single-plaintiff settlement, all of it is identifiable on its face. You cannot Safe-Harbor a one-patient narrative and keep it useful. De-identification works for population-level analytics, not for the document-level legal work that fills an in-house queue.

There is also a re-identification trap regulators flag. Modern AI is good at linking "anonymized" data back to a person by triangulating against public records, so a sloppy redaction that leaves quasi-identifiers in place can fail Safe Harbor's "no actual knowledge" test.

State overlays that change the bar

Federal HIPAA is the floor. State health-privacy laws are where in-house counsel get blindsided.

California Confidentiality of Medical Information Act (Cal. Civ. Code § 56 et seq.). Broader than HIPAA on certain disclosures. CMIA reaches providers, employers, and contractors handling "medical information," and the private right of action with nominal damages of $1,000 per violation under § 56.36 makes plaintiffs' counsel pay attention even where actual harm is hard to prove.

California appellate authority has been mixed on stretching CMIA to web-tracking theories, but the statute remains the operative reference for any vendor that touches identifiable medical records inside a privileged memo.

Texas HB 300, codified at Tex. Health & Safety Code § 181. Texas's "covered entity" definition is broader than HIPAA's. Section 181.101 sets a 60-day employee training requirement on PHI handling.

Section 181.201 sets civil penalties that scale with intent, including significantly higher penalties for knowing or intentional disclosure for financial gain. A vendor working with Texas-based clients should sit inside this training regime, not outside it.

New York SHIELD Act, N.Y. Gen. Bus. Law § 899-bb. Imposes reasonable administrative, technical, and physical safeguards on any business holding the private information of a New York resident, including health-related data. The Attorney General has used SHIELD to extract settlements where vendor oversight was thin.

Washington My Health My Data Act, Chapter 19.373 RCW. In force since March 31, 2024 (small businesses since June 30, 2024). MHMDA's definition of "consumer health data" is dramatically broader than HIPAA's PHI, sweeping in inferences, location data near a health facility, and biometric data.

A violation of MHMDA is a per se violation of the Washington Consumer Protection Act, which exposes the violator to AG enforcement and to a private right of action under the CPA. Any vendor handling client data tied to Washington consumers needs to be evaluated against MHMDA's consent and data-sale prohibitions, not just HIPAA.

Nevada, Connecticut, and a growing cluster of states have layered similar regimes on top of HIPAA. None preempt HIPAA. The vendor needs to clear whichever bar is highest for the data at issue, which in practice means HIPAA plus the strictest state regime any of your client populations live in.

How healthcare procurement actually sequences this

The way a healthcare in-house team runs this evaluation, when they run it well, does not match the vendor's preferred sales motion. Intake gets routed through privacy, not procurement.

The first artifact requested is not a demo, it is the vendor's BAA, sub-processor list, and HITRUST or SOC 2 report. If the vendor cannot produce all three in the first week, the file closes.

Week one is where most healthcare deals die. The vendor's sales engineer wants to show the product; the privacy office wants the BAA. Healthcare is one of the few segments where the demo is genuinely irrelevant until the paper is on the table.

Vendors that pre-load a BAA, a HITRUST letter, and a sub-processor table in the first email close. Vendors that treat the BAA as a "Q4 procurement workstream" die in week one.

Security and legal split ownership cleanly. Security owns the technical risk analysis (the § 164.308(a)(1)(ii)(A) artifact, the SIEM integration, the audit log review). Legal owns the BAA negotiation and the indemnification carve-outs.

The redlines that stall fastest are the ones where security and legal argue the same point through the BAA in different language. A representative stall list:

• Breach SLA (vendor wants 60 days under § 164.410's ceiling, you want 24/72)
• HIPAA penalty indemnity (vendor wants it inside the general cap, you want it carved out and super-capped at a meaningful multiple)
• Sub-processor objection window (vendor wants "commercially reasonable," you want 30 days)
• Return-or-destroy timeline on termination (vendor wants 60 days, you want 30, with a "block instead" carve-out only where deletion is genuinely impractical)
• Audit right (vendor wants "in lieu of audit, we provide SOC 2," you want a real audit right with a 12-month cap)

The single thing most in-house counsel miss on the first redline: the indemnification carve-out for HIPAA penalties. The vendor's MSA caps liability at 12 months of fees. The BAA goes silent on indemnity, defaulting back to the MSA cap. A six-figure OCR resolution on the vendor's failure to maintain its risk analysis then comes out of your budget, not theirs.

One sentence fixes it: HIPAA violations and OCR penalties are carved out of the general cap, super-capped at a meaningful multiple, or uncapped for willful or grossly negligent breach. If the vendor will not move on that sentence, the deal is dead. Find out cleanly in week two instead of week ten.

Try Vaquill AI on your matterBook a demo

The seven questions that actually screen the market

Send this list to every vendor on your shortlist before the second demo. Speed and shape of response matter as much as the answer.

1. Will you sign our BAA, or do you require yours? Right answer: "we will sign yours, or negotiate against ours, with a 30-day target." A vendor without a BAA on the shelf in 2026 has not sold into healthcare, and the deal will die in week one anyway.

2. Show me every sub-processor and the BAA status of each. Hosting region, function, BAA effective date. Special focus on the LLM provider: which contracting entity, which endpoint, which region. A vendor that cannot produce the list within a day has not mapped its own data flow, which is exactly the condition that produced the BST settlement.

3. Is PHI tokenized, redacted, or encrypted before it hits the LLM? "End-to-end TLS" is table stakes, not an answer. The signal is field-level redaction or deterministic tokenization of identifiers before the LLM call, plus customer-managed keys for the embedding store. Vendors who have built this describe the pipeline in two minutes.

4. Is US-only residency contractual or aspirational? Trust-page commitments are not contractual. The DPA-side companion to this question lives in legal AI vendors with signed DPAs and EU/US hosting.

5. What is your audit log retention period, and can we export to our SIEM? HIPAA Security Rule § 164.312(b) requires audit controls. A 90-day floor with customer-accessible export is the floor of a healthcare-ready product. Logs held internally with no export path are a posture that does not survive OCR's investigative process.

6. What is your breach notification SLA in the BAA? Push for 24 hours preliminary, 72 hours detailed. Accept 7 days only with a documented rationale. Reject 60 days outright. A vendor that defends the 60-day window is telling you it has not built the incident response apparatus you need it to have.

7. SOC 2 Type II plus HITRUST? SOC 2 Type II is the floor. HITRUST r2 is the healthcare-grade signal that the vendor has been audited against HIPAA-mapped controls. The vendors that hold HITRUST r2 built for healthcare from day one and priced accordingly. The vendors that hold only SOC 2 Type II are treating healthcare as an upsell.

Score each green, yellow, or red. Two reds and the vendor drops. Five greens earns the second demo.

What a real redaction pipeline looks like (question 3, worked)

Question 3 is the one vendors bluff most. Here is the difference between an answer and a slogan, on a single line of input.

Raw prompt the lawyer types:

"Summarize the liability exposure in Maria Gutierrez's chart (MRN 4471209, DOB 03/14/1982) for the fall on 2025-11-02 at Sunrise Memorial."

What a vendor with no real pipeline sends to the LLM: the same sentence, names and MRN intact. That is PHI leaving your boundary to an endpoint your BAA may or may not cover.

What a vendor with field-level tokenization sends:

"Summarize the liability exposure in [PERSON_1]'s chart (MRN [ID_1], DOB [DATE_1]) for the fall on [DATE_2] at [FACILITY_1]."

The model reasons over tokens, returns the analysis, and the vendor re-hydrates the names on the way back inside your tenant. Ask the vendor to walk you through this on one sentence in the demo. If they cannot produce the tokenized string in two minutes, the pipeline does not exist yet.

Six vendor categories that meet the bar in 2026

1. General-purpose legal AI at the enterprise tier with a healthcare BAA. The leading general-purpose vendors (Harvey, Spellbook, CoCounsel and similar incumbents) typically gate BAA availability behind an enterprise contract, not the self-serve plan. The practical screen: whether the vendor will name its model-provider BAA chain on a sales call. "OpenAI Enterprise, BAA in force, deployed in this region" is in the category; "we have a BAA program" is not.

2. Healthcare-vertical legal AI. A smaller set of vendors built specifically for healthcare use cases: regulatory tracking, payer-provider contract review, IRB workflow. HIPAA was the product spec, not a procurement obstacle. Ask for the HITRUST r2 letter and the date of the most recent risk analysis; both should arrive day one. Trade-off: feature breadth.

3. In-house build on HIPAA-eligible infrastructure. RAG over a controlled corpus, hosted on a HIPAA-eligible cloud region (AWS, Azure, or GCP all offer HIPAA-eligible services under their respective BAAs), with the legal team owning the pipeline. No third-party vendor in the middle of the BAA chain.

Viable at health systems with mature security engineering, increasingly common at academic medical centers.

4. LLM provider direct. OpenAI offers a BAA through its enterprise agreements; Azure OpenAI surfaces a BAA through the Microsoft Online Services Data Protection Addendum; Anthropic offers BAAs for enterprise customers using its API or AWS Bedrock. Shortest BAA chain in the market, cleanest data path, highest engineering burden.

5. CLM with a healthcare module. Ironclad, LinkSquares, and similar enterprise CLMs ship healthcare-oriented configurations with BAAs at the enterprise tier. The natural fit for a payer or provider with high inbound BAA volume that needs the CLM to manage the BAAs themselves as a contract type. Pair with a separate tool for open-ended research and drafting.

6. eDiscovery and document review vendors with BAAs. Disco, Reveal Brainspace, and similar review vendors have offered BAAs for years because litigation involving healthcare data required it. Document review at OCR-investigation scale is a known workflow with mature BAA paper. Useful when an investigation letter lands and you need to review fifty thousand documents in two weeks.

The category that does not meet the bar, ever, is the consumer LLM. ChatGPT consumer, Claude consumer, Gemini consumer. None ship a BAA.

Anything pasted into those products that includes PHI is, on its face, a disclosure to an entity that has not agreed to safeguard it under HIPAA. The fact that it happens daily in hospitals does not make it lawful. It makes it the next OCR case.

What the defensible posture looks like

Six months in, a defensible healthcare in-house legal AI posture looks roughly like this. Three vendors on contract: one general-purpose at the enterprise tier with a BAA, one CLM with a BAA, one document review tool with a BAA for investigations. Every BAA is the customer-paper version, signed in under 30 days from intake.

The risk analysis names each vendor, each sub-processor, each LLM endpoint, with a quarterly review cadence. Audit log exports flow into the security team's SIEM. Consumer LLMs are blocked at the firewall for staff handling clinical data, with an exception path through legal. The breach notification SLA in every BAA is 24 hours preliminary, 72 hours detailed.

None of that is exotic. It is the posture OCR expects on the day the investigation letter arrives.

The deputy GC who killed the deal last quarter was not being paranoid. She was applying the only test that survives healthcare procurement: ask for the BAA first, read the seven answers, watch how the vendor behaves while you do it.

Marketing writes the trust page. Lawyers write the BAA. Only one is enforceable, and the OCR resolution agreements quote the contract, not the trust page.

Action item for next week, run by the deputy GC or privacy officer: pull a one-page spreadsheet of every legal AI vendor your team uses or is evaluating. Three columns: BAA status (signed, pending, not offered), model-provider BAA (named entity, effective date, region), HITRUST or SOC 2 Type II posture.

Add every entry into your § 164.308(a)(1)(ii)(A) risk analysis with a 30-day deadline. Anything still "not offered" on day 31 comes off the stack, with a written exception path through the GC for the small number of cases where the workflow does not actually touch PHI.

That spreadsheet is the artifact OCR will ask for if an investigation lands. Build it before they ask.

FAQ

What is legal AI for healthcare? It is any AI drafting, research, or contract-review tool that an in-house legal team can lawfully point at protected health information. The bar is higher than ordinary legal AI: the tool needs a signed Business Associate Agreement, a BAA chain that reaches the underlying model provider, and HIPAA-grade infrastructure. Without all three, the tool can only touch de-identified data.

Is ChatGPT HIPAA compliant for in-house counsel? Not on the consumer tiers. OpenAI signs a BAA only for the API with Zero Data Retention and for sales-managed ChatGPT Enterprise and Edu accounts; Free, Plus, Team, and self-serve Business are excluded (HIPAA Journal, updated 2026). Pasting PHI into consumer ChatGPT is an unauthorized disclosure on its face, the same failure mode OCR settled for $500,000 against Advanced Care Hospitalists in 2018.

Does an AI vendor have to sign a BAA? Yes, if the tool creates, receives, maintains, or transmits PHI on behalf of a covered entity, it is a business associate under 45 C.F.R. § 160.103 and a written BAA is required before any PHI flows. The "conduit" exception does not apply, because an AI model uses the data to produce an answer rather than merely transmitting it.

Can I use AI on patient data without a BAA? Only if the data is no longer PHI. De-identify it under 45 C.F.R. § 164.514 by Safe Harbor (strip all 18 identifier categories) or Expert Determination (a statistician certifies very small re-identification risk). For document-level legal work on a named patient, de-identification usually destroys the document's usefulness, so a BAA-backed tool is the realistic path.

What is the difference between HIPAA-eligible and HIPAA-compliant? HIPAA-eligible means the infrastructure can be configured to meet HIPAA, for example an AWS, Azure, or GCP service that falls under that cloud's BAA. Compliant means a signed BAA plus the safeguards are actually in place and configured correctly. The eligibility is the vendor's; the compliance is shared, and the misconfiguration risk sits with your organization.

Which state privacy laws stack on top of HIPAA for AI tools? California's CMIA, Texas HB 300, New York's SHIELD Act, and Washington's My Health My Data Act all reach health data and none preempt HIPAA. MHMDA's "consumer health data" definition is far broader than PHI, and a violation is a per se Washington Consumer Protection Act violation. The vendor has to clear whichever bar is highest for the populations your data covers.

For the contract-side work once the BAA paper clears, Vaquill AI pairs AI drafting with compliance checks that flag where a draft drifts from HIPAA and the relevant state overlays. Vaquill AI offers a BAA on request for teams handling PHI in their matters and runs on US data residency with no training on your data; the security and BAA posture is documented here. You can also see how the compliance check works.

For the DPA-side companion on legal AI vendor review for in-house counsel, see the DPA review field guide. For the broader in-house buyer's playbook, see /topics/in-house-counsel.

Arshita Anand

Co-Founder & CEO · Attorney

Arshita leads product and strategy at Vaquill, building the legal AI suite that solo, small-firm, and in-house US lawyers use to run a matter end to end.