Updated August 18, 2026
TL;DR: Secure AI deployment architecture for regulated enterprises requires a local control plane that enforces runtime policy inside their own infrastructure, not an external gateway that routes telemetry to vendor servers. Three deployment models cover the full isolation range: self-hosted (physical), cloud VPC (Virtual Private Cloud), and air-gapped. All three keep data, governance logic, and audit logs inside your perimeter. The right choice depends on your network isolation requirements, update cadence tolerance, and hardware constraints. In every model, policy enforcement logic must live inside your boundary, not route to external servers. An AIUC-1 assessor, an OCC, FDIC, Federal Reserve, or NCUA examiner reviewing a financial services AI program, or a security review of a defense-adjacent deployment will ask where enforcement happened and where the evidence record lives, and both answers need to point inside your own infrastructure.
External AI security tools that route policy checks to vendor infrastructure create a data egress vector that security teams and AIUC-1 assessors will identify immediately. The telemetry generated by those checks leaves your perimeter along with the AI traffic they were meant to protect, and that is the finding that stalls most enterprise AI initiatives at security review.
This guide breaks down the infrastructure trade-offs of self-hosted, cloud VPC, and air-gapped control planes, covering data residency, governance logic placement, audit log generation, and runtime policy enforcement. The goal is a recommendation you can take into a security review with specific framework evidence behind it.
Infrastructure decisions are not separable from risk and compliance posture. Where the control plane runs determines where enforcement happens, where audit logs are generated, and whether telemetry can be produced as evidence in a defensible, internally controlled record.
| AIUC-1 pillar | Specific control requirement | Prediction Guard implementation |
|---|---|---|
| Data and Privacy | PII (personally identifiable information) leakage prevention, cross-customer isolation | PII detection and masking enforced on every model or agent call |
| Security | Runtime protection against adversarial inputs | Prompt injection defense intercepts inputs before model processing |
| Safety | Harmful output prevention | Toxicity filtering applied to outputs before they reach downstream systems |
| Reliability | Hallucination prevention, tool call restriction | Grounding verification intercepts unsupported outputs before they reach downstream systems. Access controls define which agents may call which tools and MCP servers, configured in the Admin Console. |
| Accountability | Audit logs and logging | Structured audit logs generated as enforcement evidence, SIEM-formatted inside your environment |
| Society | Prevention of AI-enabled cyber attacks and catastrophic misuse | Input and output enforcement blocks inputs designed to elicit harmful or misuse-enabling content before model processing completes |
AIUC-1, the cross-framework AI governance standard with published crosswalks to the NIST AI RMF (National Institute of Standards and Technology AI Risk Management Framework), EU AI Act, OWASP (Open Worldwide Application Security Project) LLM Top 10, ISO/IEC 42001, and others at aiuc-1.com/crosswalks, organizes requirements across six pillars. The table below maps each pillar to the specific control plane capability that addresses it.
The EU AI Act is one of the frameworks AIUC-1 maps to, and its obligations for high-risk AI systems have direct structural implications for where audit logs must be generated and retained.
The EU AI Act Article 16 activates provider obligations for high-risk AI systems, including requirements for documentation (Article 18) and logging (Article 19), with post-market monitoring addressed separately under Article 72. Article 19 requires providers to keep automatically generated logs for a period appropriate to the intended purpose of the system, and for at least six months, which makes their physical location a contractual question the deployer cannot defer to a vendor. Organizations deploying high-risk AI systems must ensure log accessibility and accuracy are under their own control. A self-hosted control plane keeps that question inside the deployer's own infrastructure by design, generating and retaining audit logs within the organization's own environment rather than routing them to external vendor servers.
Routing telemetry through an external vendor for policy evaluation means session-level AI data crosses a network boundary into infrastructure you do not control before the record is written. For a market surveillance authority reviewing your conformity documentation, that is a direct residency concern, not a theoretical risk.
The control plane handles policy checks, routing logic, and audit log generation. The data plane handles model inference, which may run on GPU (graphics processing unit) or CPU (central processing unit) depending on workload type. These two planes operate independently, and that separation matters for both latency and governance.
Prediction Guard built the control plane to run CPU-only, enforcing governance policies regardless of what hardware the model inference servers use. The control plane does not compete for GPU cycles, which means governance enforcement capacity and model inference capacity scale independently.
The governing distinction in AI architecture is not between "has a policy" and "has no policy," it is between policy enforced at the system level and policy documented in a wiki. The control plane intercepts every model or agent call, evaluates it against the governance policy defined through the control plane configuration interface, and either allows it, blocks it, or rewrites it before the response returns to the calling application. The audit log records that enforcement happened, as evidence of the action, not as a substitute for it.
Retrospective log analysis cannot satisfy runtime enforcement requirements for regulated workloads. By the time a security tool flags a violation in a log, regulated data may have already propagated to a downstream system. Practical AI episode 332 covers how increasingly complex agentic architectures create new security risks, prompt injection, data exfiltration, and tool misuse, and why guarding against these threats requires attention at the architecture level rather than after the fact.
The choice between deployment models reflects the required level of network isolation, acceptable update cadence, and available hardware infrastructure. All three models keep governance logic and audit log generation inside your environment.
| Metric | Self-hosted (physical) | Cloud VPC | Air-gapped |
|---|---|---|---|
| Network isolation | Physical network boundary | Virtual private subnet | Zero outbound internet |
| Update mechanism | Internal CI/CD (continuous integration/continuous delivery) pipeline | Cloud-native automation | Signed physical media |
| Control plane hardware¹ | CPU-only | CPU-only | CPU-only |
| Model inference hardware | GPU or CPU | GPU or CPU | GPU or CPU |
| Data egress path | No outbound network path to external infrastructure | No outbound path within VPC boundary; cloud provider infrastructure boundary applies | No outbound network path exists at any layer |
| Operational toil | High | Moderate | Highest |
¹ The control plane is CPU-only and does not compete for GPU cycles. The degree to which governance enforcement and model inference capacity scale independently will vary depending on workload type and inference configuration.
A self-hosted deployment runs the control plane on customer-owned physical hardware inside the customer's data center. All model calls, policy checks, and audit log writes happen within that physical boundary. The engineering team manages the hardware lifecycle and update process, but governance logic and enforcement remain completely under organizational control. The control plane is CPU-only, so governance enforcement runs without competing for GPU cycles, which means hardware provisioned for this architecture can allocate GPU capacity entirely to model inference servers.
A cloud VPC deployment places the control plane inside a customer-controlled virtual private subnet on AWS (Amazon Web Services), Azure, or GCP (Google Cloud Platform). The customer retains control over the VPC configuration, the data that flows through it, and the audit logs Prediction Guard generates. Cloud scalability applies to model inference servers independently of the CPU-only control plane.
An air-gapped deployment operates on a network with zero outbound internet connectivity. Every dependency, model weight, container image, and policy configuration must be present inside the enclave before deployment. Watch EP02 for a practical walkthrough of self-hosted and air-gapped architecture in manufacturing and logistics contexts.
Self-hosted, physical deployments are the right architecture when data sovereignty is a board-level requirement or when regulated data categories, including HIPAA-covered health information or other data your security and legal teams have determined cannot transit any network boundary outside the organization's direct control, must remain entirely within your own infrastructure.
Prediction Guard's control plane requires CPU compute only. Model inference servers run on GPU or CPU depending on the workload. Minimum resource requirements for production deployments vary by workload. Contact the Prediction Guard team during the deployment scoping call to confirm sizing requirements for your specific environment.
Physical isolation provides strong data residency guarantees: no byte of regulated data transits an external vendor server because there is no network path to one. Every AI input, model output, tool invocation, and policy decision completes entirely within the physical data center boundary.
Physical isolation removes the question of third-party access entirely: there is no vendor control plane, and no vendor telemetry channel, for regulated data to reach.
ISO/IEC 42001 Annex A Control A.6.2.8 (AI System, Recording of Event Logs) requires event log recording throughout the AI system lifecycle, capturing prompts, tool invocations, outputs, and affected resources as a replayable trace bound to users, sessions, and data sources.
Prediction Guard generates structured audit log output as a byproduct of active enforcement, formatted natively for Splunk, Datadog, Grafana, or any syslog-compatible target inside your environment. Prediction Guard does not hold SIEM (Security Information and Event Management) credentials, store API keys, or test SIEM connections. Prediction Guard configures output formatting so your existing ingestion pipeline handles delivery entirely within your own infrastructure. Storage, retention, and search remain your SIEM's responsibility.
For self-hosted deployments, that entire chain operates within the physical network. An ISO/IEC 42001 certification body reviewing your event log controls will find records generated, formatted, and retained inside the same boundary as the AI system they document. See EP04: OWASP Guidance Implementation for how event logging maps to security control requirements in production environments.
Self-hosted physical deployments require the most internal operational investment of the three models. Hardware procurement, lifecycle management, and capacity planning are entirely internal responsibilities. Software updates travel through your own CI/CD pipeline rather than applying automatically. Security patches require internal validation, staging, and promotion.
These are real costs. The trade-off is complete control over every byte of regulated data and governance logic. For workloads where the regulatory or contractual cost of data egress exceeds the operational cost of running internal infrastructure, self-hosted physical deployment is not a compromise, it is the minimum viable architecture.
Cloud VPC deployments fit organizations whose regulated workloads already run in a cloud-native environment and whose priority is maintaining network isolation without the hardware lifecycle burden of physical data center infrastructure.
Isolating AI traffic within a VPC requires network configurations that restrict inbound traffic to VPC-originating sources and routing configurations that keep AI application traffic, governance control plane operations, and model inference workloads within the virtual private subnet boundary. No traffic from the AI application, the governance control plane, or the model servers routes to the public internet.
Cloud region selection determines data residency for VPC deployments. For EU AI Act-compliant deployments of high-risk AI systems, data residency requirements point toward EEA (European Economic Area) cloud regions, with audit logs written and retained within the same region. Note that provider selection also matters for compliance posture: the US CLOUD Act applies to US-headquartered cloud providers globally regardless of where their regions are located, which is a structural consideration for any workload where data sovereignty requirements restrict access by foreign jurisdictions. Region and network architecture together determine compliance posture, but neither substitutes for assessing your specific regulatory obligations.
The integration path for developers is a single configuration change. Existing OpenAI-compatible and Anthropic-compatible SDK calls work unchanged. Only the base_url points at the control plane endpoint inside the VPC.
import os from openai import OpenAI # Developers only change the base_url to point to the local control plane client = OpenAI( base_url="https://prediction-guard.local.network/v1", api_key=os.environ.get("PREDICTION_GUARD_API_KEY") ) response = client.chat.completions.create( model="your-chosen-model", messages=[{"role": "user", "content": "Process regulated data."}] ) Security and GRC (governance, risk, and compliance) teams configure enforcement through the Govern page of the Admin Console, independently of the application code. Developers ship features. Security teams configure policy. Prediction Guard enforces that policy on every call regardless of which framework the developer chose. The unified AI systems page details how this separation of duties reduces friction in regulated enterprise environments.
Because the control plane is CPU-only, scaling model inference capacity (GPU-intensive) is entirely independent of scaling governance enforcement capacity. Auto-scaling groups for model inference servers respond to throughput demand without touching the control plane configuration. Governance policy stability is not affected by inference load spikes. EP10: The "USB-C" of AI covers how this composable architecture handles multi-model, multi-vendor environments under a single governed API.
Air-gapped deployments fit networks processing highly sensitive or defense-adjacent data where the organization has determined that no network path to external infrastructure is acceptable, including workloads where even a theoretical external data path is operationally or contractually prohibited. The defining characteristic is zero outbound internet connectivity at runtime. Policy checks, model calls, and audit log writes all complete without any packet leaving the enclave.
Every dependency the control plane needs at runtime must be present inside the enclave before deployment, including container images for the control plane and model servers, model weights, policy configurations, and internal libraries. The standard approach involves mirrored versions of external repositories using a private registry such as Artifactory or Nexus, with every component versioned, signed, and frozen at staging time. Every byte of every dependency must be accountable before the enclave goes live. Prediction Guard supports this workflow natively in its Kubernetes deployment model.
Updates cross the air-gap boundary on signed physical media on a controlled cadence. The workflow: stage updates on an internet-connected system, validate integrity through signature verification, transfer to the enclave, deploy to a canary subset, observe behavior, and promote to production after the observation window completes. The previous version remains available for rollback. Nothing enters the enclave over a network connection. Policy updates follow the same process.
For workloads where even a theoretical external data path is operationally or contractually prohibited, the air-gapped architecture resolves the question at the infrastructure level. Design files, technical specifications, manufacturing parameters, and other sensitive technical data processed by AI systems in this deployment model complete every operation within the enclave boundary. No AI input, model output, tool invocation, or governance decision generates a network packet that routes outside the isolated network. For security reviewers assessing defense-adjacent or mission-critical AI deployments, the relevant claim is architectural: there is no egress path, which means there is no external data transmission to assess.
Air-gapped deployments carry the highest operational overhead of the three models. Update cadence is slower and requires a manual, formally controlled process. Hardware must be provisioned with sufficient capacity for projected growth because adding capacity requires the same controlled physical access process as adding a new system. For defense and intelligence-adjacent workloads, this overhead is a regulatory and contractual requirement, not an optional trade-off.
Prediction Guard governs a heterogeneous collection of models and tools, including open-source model families, closed-vendor API endpoints, self-hosted models, and MCP (Model Context Protocol) servers, under a single governance policy configuration.
Prediction Guard applies consistent AI governance policy across every model or agent registered in an AI System, regardless of whether that model runs on local hardware, in a VPC, or behind a third-party API endpoint, including external model services such as Microsoft Azure AI Foundry and AWS Bedrock. A change to the policy configuration in the Admin Console applies to every registered model and tool on the next call. EP06: Harmonizing AI Tools covers how this unified approach eliminates the per-vendor policy drift that accumulates when teams govern each model integration independently.
The OWASP Top 10 for Agentic Applications defines ten risk categories a self-hosted control plane must address for agentic workloads. Two are directly tied to infrastructure architecture.
ASI01 (Agent Goal Hijack): Attackers manipulate an agent's objectives through prompt-based manipulation, deceptive tool outputs, or poisoned external data. The common industry mitigation at the control plane is threshold tuning and allow-listing to restrict the instruction surface available to injected content before agent behavior can be redirected. Prediction Guard addresses this through prompt injection defense, which intercepts inputs before model processing, reducing the instruction surface available to injected content at the point of entry.
ASI02 (Tool Misuse and Exploitation): Agents misuse legitimate tools due to prompt injection, misalignment, or unsafe delegation. The common industry mitigation at the control plane is permission-aware scopes per caller and argument validation at the boundary, restricting the action surface available to a misaligned or injected instruction before tool execution completes. Prediction Guard addresses this through access controls configured in the Admin Console, which define which agents may call which tools and MCP servers before execution is permitted. EP03: Agentic AI Threats covers practical mitigations for the full OWASP agentic risk categories in production deployments.
ISO/IEC 42001 Annex A Control A.6.2.8's event log recording requirement, capturing prompts, tool invocations, outputs, and affected resources as a replayable trace, applies identically regardless of whether the governed model or agent runs on local hardware, in a VPC, or behind a third-party API endpoint, and the same enforcement-generated audit log satisfies it in every case.
The deployment model decision determines where data, governance logic, and telemetry stay: inside your environment or routing through external infrastructure. That architectural boundary has direct structural implications for how audit evidence is generated, where compliance documentation originates, and what security reviewers will find when they examine your AI governance posture.
Audit readiness requires three things: a complete AI asset inventory, evidence that governance policy was enforced on the AI interactions in scope, and a record of that enforcement living inside your own infrastructure. AI System registration produces the inventory as a structured record and the AIBOM (AI Bill of Materials) as an exportable CycloneDX artifact. Prediction Guard's audit log satisfies the evidence requirement. For an AIUC-1 assessor, an OCC, FDIC, Federal Reserve, or NCUA examiner reviewing a financial services AI program, or a security review of a defense-adjacent deployment, the combination represents a complete, internally controlled governance record.
The NIST AI RMF implementation playbook covers how to structure that evidence for specific framework function requirements.
The build-vs-buy calculation for governance infrastructure is consistently more expensive than engineering teams estimate. Deploying Prediction Guard's control plane inside existing Kubernetes infrastructure, with transparent integration via a base_url change and Admin Console configuration, covers runtime enforcement, SIEM integration, and standards-aligned policy coverage. The scaling agentic AI blog post covers the cost and governance trade-off analysis at enterprise scale.
Prediction Guard's model-agnostic control plane governs open-source model families, closed-vendor API endpoints, and self-hosted models under one policy framework. Swapping the underlying model, whether due to a cost change, a capability upgrade, or a vendor deprecation, does not require rebuilding governance policies. The governance policy configuration in the Govern page of the Admin Console applies to whatever model is registered in the AI System. EP07: Evaluating AI Model Performance covers how model selection decisions interact with governance architecture requirements at the practitioner level.
Runtime policy enforcement overhead is small relative to model inference latency and tool execution costs, with the specific figure varying based on deployment configuration and workload type. Because the control plane is CPU-only, governance enforcement capacity scales independently of inference capacity, so policy checks do not compete with model serving for GPU resources as call volume increases.
The following questions represent the specific scoping points that typically arise when presenting a deployment architecture recommendation to a security review.
In all three deployment models, Prediction Guard generates structured audit log output as a byproduct of enforcement. In self-hosted and VPC deployments, logs route to your existing SIEM ingestion pipeline via output formatting configured on the Monitor page of the Admin Console. In air-gapped environments, that same pipeline operates entirely within the isolated network. Prediction Guard does not hold SIEM credentials or store logs.
Noma's Kong Gateway plugin, by contrast, routes telemetry to api.noma.security on port 443 for centralized security oversight, as documented in the Kong developer plugin reference. For regulated organizations with data residency or sovereignty requirements, that external routing is a structural incompatibility. The Noma Security assessment covers this trade-off in detail. Prediction Guard generates equivalent audit log evidence inside your own environment.
In organizations running multiple isolated deployments, consistent enforcement requires the same governance policy configuration applied to each control plane instance. The Admin Console configuration is the single source of truth for what gets enforced. Deploying the same configuration across multiple instances produces identical enforcement behavior regardless of location. Audit logs generated by each instance are formatted identically for SIEM ingestion, so your security operations team sees a consistent event schema across every governed AI deployment in the organization.
Ready to assess which deployment model fits your infrastructure and risk and compliance requirements? Book a deployment scoping call to work through the trade-offs for your environment.
Prediction Guard's control plane is CPU-only and does not require GPU resources, so GPU capacity can be allocated entirely to model inference servers.
Yes. Policy configurations carry across self-hosted, VPC, and air-gapped environments, so migration changes the infrastructure target without requiring the governance policy to be rebuilt.
In air-gapped environments, Prediction Guard formats audit logs natively for Splunk, Datadog, Grafana, or generic syslog and outputs them to a local ingestion pipeline inside the isolated network. Prediction Guard does not hold SIEM credentials or test SIEM connections, so the integration works identically in isolated and connected environments, with your SIEM infrastructure handling all delivery and retention.
Prediction Guard addresses the OWASP Top 10 for Agentic Applications through runtime enforcement mechanisms that intercept and validate inputs before model processing and evaluate permissions before actions complete, which addresses ASI01 (Agent Goal Hijack) and ASI02 (Tool Misuse and Exploitation) directly.
SIEM: A centralized system that aggregates, stores, and analyzes security event logs from multiple sources to enable threat detection, incident response, and compliance reporting.
Sovereign AI control plane: A self-hosted governance system that runs inside your own infrastructure to compose, secure, and govern AI systems without data egress.
Grounding verification: A probabilistic check that intercepts unsupported outputs before they reach downstream systems by verifying generated content and agent actions against trusted data sources. This check is probabilistic, not deterministic, and does not guarantee accuracy on every inference.
AI System registration: The process of cataloging models, tools, MCP servers, and datasets within the control plane to enable governance enforcement and generate an exportable AI Bill of Materials (AIBOM) in CycloneDX format.
AIBOM: The exportable inventory artifact produced by AI System registration, structured in CycloneDX format to track model provenance, tool dependencies, and dataset references for audit and supply chain accountability.
Control plane: The infrastructure component that handles policy enforcement, routing logic, and audit log generation for every AI interaction. Prediction Guard's control plane is CPU-only.
Air-gapped deployment: A deployment architecture in which the control plane, model inference servers, and all dependencies operate on a network with zero outbound internet connectivity, with updates transferred via signed physical media on a controlled cadence.
AIUC-1: A cross-framework AI governance standard organized across six pillars (Data and Privacy, Security, Safety, Reliability, Accountability, Society) with published crosswalks to NIST AI RMF, EU AI Act, OWASP LLM Top 10, ISO/IEC 42001, and other frameworks at aiuc-1.com/crosswalks.