TL;DR: No architecture is HIPAA compliant on its own, but a HIPAA-compliant AI agent architecture makes minimum necessary access, audit controls, and business associate boundaries enforceable and provable. This guide lays out five layers for healthcare AI teams automating clinical workflows: a unique identity and delegated authority for each agent, runtime controls on what moves through models and tools, Immutable Audit Logs as evidence, containment and hard limits, and a boundary that keeps protected health information inside infrastructure you control. Nothing here is legal advice, and HIPAA does not prescribe or certify any architecture.
No architecture is HIPAA compliant on its own. HIPAA compliance describes how a covered entity or business associate handles protected health information, and an architecture decides whether that handling can be controlled and proven. For HIPAA-compliant AI agents, the architecture is where minimum necessary access, audit controls, and business associate boundaries either exist or do not.
Definition. A HIPAA-compliant AI agent architecture, as this guide uses the term, is a design in which every agent action that touches protected health information runs under an identified agent with delegated, minimum necessary authority, passes through enforced controls on what goes into and out of models and tools, leaves a tamper-resistant record, can be stopped, and stays inside boundaries covered by the right agreements. HIPAA does not prescribe or certify such a design. It sets requirements that your organization has to meet and be able to show, and this design is one way to make them demonstrable.
This guide is for regulated enterprise AI teams in healthcare. It explains how to structure agents for clinical workflow automation while protecting patient data privacy: what HIPAA data security asks of an agent, a five-layer reference architecture for healthcare AI compliance, and a worked example of one agent moving through it. In practice this comes down to operational control of agents: knowing what each one may touch, being able to stop it, and being able to prove both. A disclosure: Prediction Guard publishes this guide and builds a sovereign AI control plane, which appears in the architecture. Nothing here is legal advice, and the HIPAA text and HHS guidance are cited to their own pages.
What HIPAA Asks of an Agent: HIPAA Data Security and Minimum Necessary
HIPAA has no chapter for AI agents, and it does not need one. An agent that reads, creates, or transmits protected health information is part of the systems and workflows the entity is responsible for, so the same Privacy Rule and Security Rule requirements apply that apply to workforce members and information systems.
Minimum necessary. The Privacy Rule's minimum necessary standard, 45 CFR 164.502(b), requires reasonable efforts to limit uses, disclosures, and requests for PHI to what the purpose needs, and 164.514(d) requires role-based policies for workforce access. The exemption is narrow. HHS's minimum necessary FAQ says disclosures between providers for treatment are exempt, and that does not cover payment or health care operations. Most agent automation sits on the payment and operations side, in prior authorization, claims, coding, scheduling, and quality reporting, so scope every agent to its task however clinical the setting feels.
Technical safeguards. The Security Rule's 45 CFR 164.312 sets five standards: access control, audit controls, integrity, person or entity authentication, and transmission security. Two matter immediately for agents. HIPAA requires unique user identification to support accountability for access to ePHI. For agentic systems, a strong implementation approach is to assign each agent its own identity rather than relying on shared service accounts, which makes agent activity independently attributable and allows authority to be scoped and revoked. Audit controls also require mechanisms to record and examine activity in information systems that contain or use ePHI. Agent activity within those systems should therefore be captured in the audit records.
Business associates. Agreements follow the data. HHS's cloud computing guidance says a cloud service provider that creates, receives, maintains, or transmits ePHI for a covered entity is a business associate, even if the data is encrypted and the provider holds no key. A model API that receives PHI is generally in the same position, which is one reason teams run inference inside their own boundary.
A rule still in motion. A proposed Security Rule update, published in the Federal Register in January 2025, would remove the addressable-versus-required distinction and require measures such as multi-factor authentication, encryption, and an asset inventory. As of October 2026 it is not final. The federal regulatory agenda projects final action in July 2027, though that date is a projection and not a deadline, and the current rule remains in force. Healthcare AI teams should consider these proposed measures when designing future security controls, while continuing to meet the requirements of the Security Rule currently in force.
HIPAA is also not the only regime. State health privacy laws, 42 CFR Part 2 for substance use disorder records, and FDA rules when software drives clinical decisions may apply, depending on the workflow.
Healthcare AI Compliance Reference Architecture: The Picture
These requirements do not translate into a single security feature. They require a coordinated architecture in which identity, authorization, runtime enforcement, auditability, and containment work together. The layers below are recommended implementation approaches, not an architecture prescribed by HIPAA.
Everything that touches PHI, including the audit records, sits inside one boundary, and every model and tool call passes through one control point. In this picture that control point is a sovereign AI control plane, the architecture that identifies, authorizes, enforces, meters, records, and contains agent actions, running inside your boundary:
clinicians, staff, and EHR events
|
+---------------v------------- your boundary ----------------------+
| agent runtime (sandboxed, one identity per agent) |
| | |
| control plane: identity, authority, runtime controls, |
| limits, audit, containment |
| | | | |
| model inference EHR and payer tools PHI stores |
| (self-hosted) (scoped per task) (entitlements) |
| |
| Immutable Audit Logs, SIEM, evidence store (access-controlled) |
| default-deny egress | encryption in transit | key custody |
+-------------------------------------------------------------------+Audit records stay inside the boundary and are protected like the systems they describe, since they can contain identifiers. Anything exported, such as to an external SIEM, should carry only records that have been sanitized and protected under the applicable safeguards.
Each layer does one job. Layer 1 establishes identity and authority. Layer 2 enforces what moves through the agent. Layer 3 proves what happened. Layer 4 limits the damage when something goes wrong. Layer 5 establishes responsibility across infrastructure and vendors.
Layer 1: Identity and Delegated Authority for Minimum Necessary Access
Start with who is acting. Give every agent a unique, revocable identity, so every later record has something to attach to. Then define delegated authority, the scoped, time-bound authority an action runs under, traceable to an accountable human. In HIPAA terms, delegated authority is how minimum necessary becomes enforceable: the agent can reach only the PHI its current task requires, decided at the time of each request, and not everything its credentials would allow.
The common failure is a service account with broad read access to the chart. Even if the agent only ever reads the fields it needs, nothing technical stops it from reading more, and a manipulated agent inherits all of it. Prefer narrow resource scopes per task, deny by default, and retrieval that returns the fields the task needs. Prediction Guard's guide to authorization scope by industry describes the healthcare case, PHI-scoped access with records of the tool invocations and PHI fields accessed, and the principle behind it is least agency: the minimum authority a task needs.
Layer 2: Runtime Controls for Protected Health Information
Runtime controls decide, on every call, whether it is allowed right now. They work in two layers.
Component Input/Output Controls (what passes into and out of a model or tool) handle the PHI itself. They detect identifiers in content before it reaches a component that should not see it, mask or block what the task does not need, and check outputs before they reach a user, a tool, or a log. HHS lists 18 categories of identifiers, from medical record numbers to biometric identifiers, and Prediction Guard's guide to PII detection and redaction covers how to evaluate detection for regulated data. Where a task does not need identity at all, de-identification under 45 CFR 164.514(b), by Safe Harbor or Expert Determination, takes the data outside PHI. Detection in free-text clinical notes is imperfect, so treat it as a control to test, not a guarantee.
Agent Behavior Controls (what an agent may do, in what sequence, against which systems) handle the workflow. They decide which EHR and payer tools an agent may call, whether it may read or write, and the order of steps, such as retrieve, draft, human review, then submit. They are also where you keep autonomous, consequential clinical writes off the table by default.
Layer 3: Evidence for Audit Controls
This layer proves what happened. HIPAA requires mechanisms that record and examine activity in systems that contain or use ePHI. Immutable Audit Logs (tamper-resistant records of each action and control decision) provide one way to implement that audit control requirement by making agent activity traceable and helping prevent unauthorized alteration of records. A useful record names the agent identity and the accountable human, the operation, the control that decided, and the time.
Where possible, logs should capture agent identities, operation types, record references, field names, control decisions, and timestamps rather than copying PHI values into the audit records. Any identifiers that could constitute PHI must themselves be protected under the applicable HIPAA safeguards. Each intervention, a recorded control action that blocks, restricts, pauses, or terminates an agent action, also produces an Evidence bundle (what was attempted, which control acted, and what was contained). For what a defensible record contains, see this guide to audit log implementation.
On retention, HIPAA requires the documentation the Security Rule specifies, such as policies, procedures, and required actions, activities, and assessments, to be kept for six years under 45 CFR 164.316(b)(2). That requirement does not automatically cover every operational audit event, and the rule does not state a separate retention period for audit records, so many organizations align them to the six-year documentation requirement. Confirm the right period with counsel.
Layer 4: Containment and Limits
This layer limits the damage when something goes wrong. Containment pauses, scopes down, or revokes an agent in one action. For PHI, that matters because the breach notification clock runs from discovery, and under HHS's Breach Notification Rule notices are due without unreasonable delay and no later than 60 days. A fast stop limits the damage, and the records from Layer 3 let you scope what was touched. Hard limits on volume, such as how many records an agent may touch per task, bound what a single failure can expose, as in Prediction Guard's guide to blast radius reduction.
Layer 5: The Boundary and Patient Data Privacy
Where PHI lives decides who needs to sign what. Inference on infrastructure you control can keep prompts, outputs, and logs inside your boundary, provided the deployment does not expose ePHI through telemetry, external logging, support access, or other services. This can remove the model vendor from the direct PHI processing path, though any other vendor with access to systems holding ePHI needs its own analysis. Running in your own cloud account does not remove the cloud provider from the picture, since HHS's guidance makes the provider a business associate that needs an agreement. Using a vendor API means a business associate agreement before PHI reaches it, plus clear terms on retention, training use, and subprocessors. Prediction Guard's deployment models guide compares self-hosted, cloud VPC, and air-gapped options.
Ownership closes the loop. A registry with a named owner, purpose, data scope, review date, and decommission path for every agent is how you know where ePHI flows, which HIPAA's risk analysis requirement presupposes, and unregistered agents are the ones nobody has analyzed.
Clinical Workflow Automation, End to End: A Prior Authorization Agent
A coordinator asks an agent to assemble a prior authorization request. The agent runs under its own identity, with delegated authority tied to that coordinator and limited to this one case. Because prior authorization is a payment and operations task, it retrieves only the order, the diagnosis, and the notes the request needs, not the whole chart.
Before anything reaches the model, Component Input/Output Controls mask identifiers the draft does not require, and the draft is checked on its way out. Agent Behavior Controls allow reads from the EHR and the payer portal but not submission until the coordinator has reviewed the packet. Each step leaves an entry in the Immutable Audit Logs. If the agent starts requesting records outside the case, containment revokes its authority in one action, and the Evidence bundle shows what was attempted and stopped. None of this is prescribed by name in HIPAA. It is one way of making the requirements demonstrable.
Where This Architecture Stops
An architecture supports a HIPAA program and does not replace one. Risk analysis, written policies, workforce training, business associate agreements, and breach response all sit outside it. It also does not settle clinical safety or other regimes, such as FDA rules where software influences diagnosis or treatment.
Prediction Guard's published limits apply too. It enforces on traffic routed through it, per-invocation authorization at the tool server is not yet generally available, token-based limits apply per API key today, and human approval gates for high-risk tool executions are on the roadmap, as its blast radius guide states. For clinical writes, that means keeping a human in the loop in your own workflow and asking every vendor, Prediction Guard included, what exists today.
Questions About HIPAA-Compliant AI Agents
1. Can an AI agent be HIPAA compliant?
Compliance attaches to how a covered entity or business associate uses the agent, not to the agent itself. An agent can be used in a compliant way when the Privacy and Security Rule requirements are met and you can show it.
2. Do I need a business associate agreement with my model provider?
Generally yes, before PHI reaches any vendor that creates, receives, maintains, or transmits it for you. With inference inside your boundary the model vendor may never receive PHI, but your cloud host would still be a business associate. Confirm with counsel.
3. Can PHI go in a prompt?
Where the use is permitted and the model runs inside safeguards and agreements that cover it, yes. Send only the fields the task needs, and de-identify when the task does not need identity.
4. Does self-hosting make an AI agent HIPAA compliant?
No. It removes a third party from the PHI path and keeps the evidence local, but access policies, risk analysis, training, and incident response still have to exist.
5. How long should audit records be kept?
HIPAA requires the documentation the Security Rule specifies to be retained for six years. That does not automatically cover every operational audit event, and the rule does not state a separate period for audit records, so many organizations align them to it. Confirm with counsel.
Compliance Is What You Can Show
An examiner will not ask whether your architecture has a name. They will ask what each agent was allowed to touch, who gave it that authority, and what record shows what it did. A design that can answer those three questions makes agent activity accountable, auditable, and controllable. That is what operational control of agents means when the data involved belongs to a patient.