HIPAA for AI companies

HIPAA for AI companies: scribes, LLMs and PHI

AI scribes and LLM-powered clinical tools handle PHI in places traditional SaaS does not: raw audio of patient visits, transcripts, prompts and model outputs. HIPAA has no AI-specific rule, so the existing rules apply to every one of those. This guide covers BAAs with model vendors, de-identification, retention and how your obligations flow down to subprocessors.

Key takeaways

  • If your AI product processes PHI for a provider or payer, you are a business associate and the Security Rule applies to the whole pipeline.
  • Every model, transcription and hosting vendor that receives PHI needs a BAA, and only the services or configurations that BAA covers may receive PHI.
  • Using PHI to train or improve models requires permission in your BAA or proper de-identification under 45 CFR 164.514.
  • Set and enforce retention for audio, transcripts, prompts and logs; return or destroy PHI when a customer contract ends.
  • Hospital reviewers will ask exactly where PHI goes. A clear data-flow diagram and subprocessor list answer most of their AI questions.

Where PHI goes in an AI product

Map the pipeline end to end before you answer a security questionnaire. For an ambient scribe it typically looks like this:

Each step is part of your risk analysis under 45 CFR 164.308(a)(1), and each vendor in it is a subcontractor business associate.

  • Capture: audio recorded on a clinician’s phone, browser or room device.
  • Transport and storage: audio uploaded and stored, at least until it is transcribed.
  • Transcription: a speech-to-text service, yours or a vendor’s.
  • Generation: an LLM drafts the note from the transcript, often with patient context in the prompt.
  • Review and write-back: the clinician edits the note, which may be written to the EHR through an integration.
  • Telemetry: logs, traces, evaluation sets and error reports, which often capture prompts and outputs.

BAAs with model and cloud vendors

You may only send PHI to a vendor that has signed a BAA with you, and only to the services and settings that BAA covers. The major vendors offer BAAs, but each limits coverage to specific products or configurations, and the terms change. Check the vendor’s current page and your signed agreement before routing PHI anywhere.

A consumer chat product from the same vendor is usually not covered by its API BAA. Make sure staff cannot paste PHI into tools outside your covered setup.

VendorWhat the vendor says (as of November 2026, confirm with the vendor)
OpenAIOffers a BAA for eligible API services. See OpenAI’s help article for which services and settings are eligible.
AnthropicIts BAA covers specific HIPAA-ready configurations. See Anthropic’s support article for which ones.
AWSPublishes a list of HIPAA eligible services; only services on that list are covered by the AWS BAA.

Training on PHI and de-identification

A business associate may only use PHI as its BAA permits. Using customer PHI to train, fine-tune or evaluate models requires that permission, and many hospitals will refuse it. The alternative is de-identified data, which is no longer PHI.

HIPAA recognises two de-identification methods (45 CFR 164.514(b)): Safe Harbor, which removes 18 listed identifiers and requires no actual knowledge that the rest could identify someone, and Expert Determination, where a qualified expert documents that the risk of re-identification is very small. Free-text clinical notes and audio are hard to de-identify reliably with Safe Harbor alone; voices, rare conditions and dates in narrative text are common misses.

Your BAA must also permit you to create de-identified data in the first place. Write it into the agreement rather than assuming it.

Retention of audio, transcripts and prompts

HIPAA does not set a retention period for the PHI you process; your customer contracts and state medical-record laws do. HIPAA does require that you return or destroy PHI at the end of the contract where feasible (45 CFR 164.504(e)(2)(ii)), and every copy you keep is in scope for the Security Rule and for breach notification.

  • Decide how long you keep raw audio. Many teams delete it once the note is signed, unless the customer contract requires otherwise.
  • Set retention separately for transcripts, generated drafts, prompts and logs, and make sure vendor-side retention matches.
  • Keep PHI out of observability and evaluation tools that are not covered by a BAA.
  • Document the schedule in a policy and show it to buyers; it is one of the first AI questions in a hospital review.

Subprocessor flow-down

Your obligations flow down. Each subcontractor that handles PHI for you must sign a BAA giving the same assurances you gave your customer (45 CFR 164.502(e)(1)(ii), 164.308(b)). If your customer BAA requires breach notice within five days, your vendor agreements need to support that.

Publish a subprocessor list that says what each vendor does and whether it handles PHI, and tell customers before you add one. Hospital BAAs often require that notice.

Beyond HIPAA

Health systems may ask AI vendors about bias testing, intended use, human review and model change management, and some state laws regulate AI use in health care. If you sell into Europe, the EU AI Act may apply. Shieldra supports ISO 42001, the NIST AI RMF and the EU AI Act as optional add-on frameworks for teams that need them.

In January 2025 HHS proposed changes to the Security Rule (90 FR 898), including stricter encryption, MFA, asset-inventory and testing requirements. As of November 2026, that proposal has not been finalized; the rule in force is the current 45 CFR Part 164, Subpart C. Many hospitals already ask for those controls in security reviews, so treat them as buyer expectations rather than law.

How Shieldra helps an AI health company

Shieldra keeps the evidence for your AI pipeline in one place, ready for a hospital review.

  • HIPAA Security Risk Analysis that covers each system in your pipeline, with Security Officer sign-off.
  • Vendor and subprocessor tracking for model, transcription and cloud vendors: BAA status, BAA upload, and expiring BAAs highlighted.
  • Policy templates you adopt for retention, access control and incident response.
  • Incident and breach tracking with a 60-day notification timeline.
  • A trust center to share your subprocessor list, policies and risk analysis summary with buyers.
  • Shieldra is a no-PHI service: document your pipeline in it, but do not upload audio, transcripts or other patient data.

Working toward your first hospital security review?

We are looking for a small number of health-tech design partners. We will work with you hands-on to get your first hospital or payer security review done, and your feedback shapes what we build next.

This guide is general information, not legal advice. Shieldra helps you prepare and organise evidence; it does not certify compliance, and HHS does not approve or endorse any private HIPAA credential.

Frequently asked questions

Is an AI scribe a HIPAA business associate?

Yes, if it records, transcribes or drafts notes from patient encounters for a covered entity. The audio, transcripts and drafts are PHI, so the Security Rule, the BAA and breach notification all apply.

Can we send PHI to an LLM API?

Only if the vendor has signed a BAA with you and the specific service and configuration you use is covered by it. OpenAI, Anthropic and AWS each describe which services or configurations their BAAs cover; check their current pages and your signed agreement.

Can we train our models on customer PHI?

Only if your BAA with that customer permits it. Otherwise use data de-identified under 45 CFR 164.514(b), and make sure the BAA permits you to de-identify. Many hospitals decline training rights, so plan for it.

How long should we keep encounter audio?

HIPAA does not set a period. Your customer contract and state law do. Keep the minimum you need, document the schedule, make vendor-side retention match, and return or destroy PHI at contract end.

Is there a HIPAA rule specifically for AI?

No. The Privacy, Security and Breach Notification Rules apply to AI systems the same way they apply to any system handling PHI. The January 2025 Security Rule proposal has not been finalized.