Ask most security teams whether they have PII protection for their AI systems, and the answer is yes. Ask them whether that protection would catch a customer's SSN pasted mid-sentence into a debugging prompt, or an AWS key sitting in a stack trace someone forwarded to a model, and the confidence usually drops. The most common way sensitive data leaks into an AI system is not a sophisticated attack. It is a developer pasting a log file to debug a formatting issue, a support agent copying a customer record into a summarizer, a script forwarding a database row into a prompt, with nobody checking what is actually in that text before it leaves the building. By the time anyone notices, the data has already reached a model endpoint that may log it, retain it, or use it in ways the organization never agreed to.
This guide is a practical, step-by-step path to closing that gap: how to detect and redact PII and secrets before a prompt reaches a model, and how to apply the same discipline to what the model sends back.
Before building any detection logic, get specific about what you are protecting against, because "PII" means something different to a healthcare team than it does to a fintech team. At minimum, most enterprise deployments need to cover two distinct categories:
Personal data: names, email addresses, phone numbers, physical addresses, government-issued identifiers (SSNs, passport numbers, national insurance numbers), financial account and card numbers, and health information where applicable
Secrets: API keys, access tokens, database connection strings, internal service credentials, and infrastructure identifiers that get pasted into a prompt by accident when someone is debugging or troubleshooting
Skipping this step is why so many redaction efforts end up narrow. A detector tuned only for the first category will wave through an AWS access key sitting in the middle of a stack trace, because it was never told to look for one.
Where does the check actually run? If the answer is "inside the application code that calls the model," you have a problem: any new code path, any script, any internal tool that skips that specific function call skips the check entirely.
The fix is to move enforcement to the layer every request has to pass through regardless of which part of the codebase initiated it, typically an SDK wrapper or a sidecar proxy sitting between the application and the model endpoint. Every outbound call gets inspected the same way, whether it came from the primary application, a batch job, or a script someone wrote last week. This is the specific gap Prediction Guard's PII detection and redaction work was built to close: a script that calls the model directly, bypassing whatever your application layer normally does, still has to pass through the same enforcement, because the check lives at the request layer itself rather than inside application logic that a new code path can simply skip.
PII and secrets need different detection approaches, and treating them the same is where accuracy drops. Named-entity recognition works well for personal data because it understands context, a string of digits might be a phone number or might just be a quantity, and the surrounding text usually disambiguates it. Secrets are the opposite: they are structured, high-entropy strings that follow known formats (an OpenAI key, an AWS access key, a JWT), so pattern matching combined with entropy scoring catches them more reliably than a general-purpose language model would.
Whichever method you use, test it against real, messy production text, not clean sample data. Nicknames, informal phrasing, and PII embedded mid-sentence are where detection quietly fails.
What actually happens the moment a detector flags something, does it stop the request, quietly fix it, or just make a note for later? Detection without a clear enforcement policy just produces a log of problems nobody acts on. Three responses cover most real cases, and the right one depends on the data category, not a blanket rule applied everywhere:
| Response | Use it when |
|---|---|
| Block the request | The category has zero tolerance, such as a secret or a government ID in a tool with no legitimate reason to see one |
| Redact or mask | Most incidental PII exposure from normal, careless use, where the rest of the request is still legitimate |
| Tokenize | A downstream system needs to match on the same value repeatedly without ever storing the real one |
A program that only blocks will generate enough false positives that people find ways around it. A program that only redacts will let categories through that genuinely needed to be stopped outright. Configure the response per category deliberately.
A clean prompt does not guarantee a clean response. A model can surface sensitive content pulled from a retrieval pipeline that the requesting user should not have access to, reproduce something it picked up during fine-tuning, or get manipulated by an injected instruction into paraphrasing a secret it has access to in a way that slips past a naive keyword filter. OWASP's LLM Top 10 lists sensitive information disclosure and hidden context exposure as distinct risk categories for exactly this reason, the leak does not have to originate in the user's own input.
An employee asks an internal AI assistant: "What's our current parental leave policy?"
The request contains no sensitive information. But the retrieval pipeline pulls in a policy document containing an old example email with an employee's name and salary.
The model summarizes the document faithfully, and exposes the salary in its response.
Nothing was wrong with the input. The sensitive information entered through retrieval.
Run the same detection and redaction logic on the model's output before it reaches the user or a downstream system. Prediction Guard's work on detecting exfiltration attempts embedded in prompts covers what these attempts typically look like: requests for environment variables, API keys, or configuration values disguised as an ordinary-sounding instruction, which is exactly the kind of pattern an output-side check needs to catch even when the original prompt looked harmless.
If a security review had to reconstruct exactly what happened six months from now, would there be anything to reconstruct from? Every block, mask, or tokenization decision should generate a timestamped record: what was detected, what category it fell into, what action was taken, and which policy version was in effect at the time. Without this, a security review after an incident becomes archaeology instead of a quick query.
A properly built evidence pipeline hashes each entry at generation and writes it to append-only storage, so the record can't be quietly edited after the fact, and the logs should flow into infrastructure your security team already monitors rather than staying locked inside a vendor's dashboard.
Redaction that only catches PII typed in plain text misses how people, and attackers, actually try to get around it. Test your pipeline against the cases that break naive filters:
If your detection only passes a clean benchmark and has never been run against these patterns, treat that as an open gap, not a finished implementation.
This is the step that gets skipped most often, and it undoes everything above it when it does. If the redaction check itself works by calling a third-party API, the sensitive data has to leave your infrastructure to be evaluated, which means the control meant to stop a leak has become one.
Prediction Guard treats this as a deployment decision rather than a pricing tier: the same detection and redaction logic runs identically whether it is deployed on-premises, in a private cloud VPC, or fully air-gapped, so the data being inspected never has to leave the network it originated in for any step of the pipeline, not just the ones that felt convenient to keep local. A program that only locks down its most sensitive workload this way usually has quiet gaps everywhere else. Not every workload needs the strictest version of this, and a framework for deciding when self-hosting is actually required is worth working through before defaulting to whichever option is fastest to stand up.
Responsible AI deployment in a regulated environment is not defined by whether an organization has PII detection. Most do by now. It is defined by whether that detection covers secrets as well as personal data, runs on responses as well as prompts, and operates inside a boundary the organization actually controls rather than one it has to trust a vendor with.