How to fix: sensitive data leaking into prompts
Last verified: June 2026· Teams using hosted LLM APIs
Where this shows up
Teams using hosted LLM APIs
How it shows up
The same root cause surfaces in a few different ways. If any of these match, you're in the right place.
- Customer records, internal documents or source code visible in prompt payloads stored in a tracing or logging platform.
- API keys or access tokens appearing in prompts because they were present in a context object that was serialised wholesale.
- Trace stores readable by a much wider internal audience than the database the data originated in.
- Retention on prompt payloads inherited from general application logging, often far longer than anyone would have chosen deliberately.
Diagnose it first
Confirm which cause you actually have before changing anything — most wasted time here comes from fixing the wrong one.
| Check | What the result tells you |
|---|---|
| Sample stored prompt payloads and read them as an auditor would. | This is uncomfortable and quick, and it is the only reliable way to find out what is actually in there. Assumptions about what gets sent are wrong more often than not. |
| Trace how context objects are assembled into prompts. | Leakage usually comes from serialising a whole object — a user record, a request context, a configuration — where only two fields were needed. The extra fields travel silently. |
| Compare who can read the trace store with who can read the source system. | If the trace store is broader, you have created a second, less-governed copy of the data. This is the single most common finding in this area. |
| Check retention on payload text separately from numeric telemetry. | Numeric fields are cheap and useful for a long time. Raw prompt text is rarely justified beyond weeks, and is very easy to justify against you later. |
The fix
- 1Inventory where prompts are sent and under what data agreements.
- 2Add redaction at the boundary to strip secrets and PII before requests leave your network.
- 3Set and enforce a data-residency policy in tooling, not just documentation.
- 4For regulated data, use self-hosted inference so nothing sensitive leaves your environment.
- 5Audit logs and prompts for what’s actually being sent — assumptions are usually wrong.
Provider-specific behaviour
The error means roughly the same thing everywhere, but each provider signals it differently.
- Two separate destinations
- Prompt content goes to the model provider and, independently, into your own observability stack. These have different controls, different retention and different audiences, and a policy that addresses only the first misses where most of the practical exposure sits.
- Redaction belongs in the instrumentation layer
- Applying it before anything leaves the process is enforceable; applying it as a cleanup job on the store means the unredacted version existed, was replicated, and may have been read.
What doesn't work
Common fixes for this error that look right and aren't.
Writing a policy asking engineers not to put sensitive data in prompts.
The failure is inadvertent rather than deliberate — a serialised object, an over-broad retrieval result — so guidance does not prevent it. One enforcement point does.
Turning off payload capture entirely.
It removes the exposure and most of your ability to debug, so it gets quietly re-enabled during the next incident and left on. Tiered capture — structural in production, full in development — is what survives.
Relying on a detector to catch everything.
Pattern detectors catch keys and well-formed identifiers and miss free-text customer data entirely. They are a useful mitigation and a poor primary control.
Prevent it
Treat every prompt as an egress channel: redact at the boundary, enforce residency in code, and self-host where the law requires it.
Common variations and related errors
You'll usually hit this same root cause under a few different names. Same fix.
- "PII in prompt"
- "secrets logged to LLM provider"
- "customer data sent to model"
- "sensitive data in request body"
Frequently asked questions
What causes “sensitive data leaking into prompts”?
Secrets, PII, or proprietary data are being sent to a model provider inside prompts, often unintentionally.
How do I prevent “sensitive data leaking into prompts” from recurring?
Treat every prompt as an egress channel: redact at the boundary, enforce residency in code, and self-host where the law requires it.
How does sensitive data end up in a prompt in the first place?
Almost always by accident of construction rather than intent. A context object is serialised wholesale when only two fields were needed; a retrieval step returns a document with more in it than the question required; an error handler includes a request body for debugging. None of these look like data handling decisions in review, which is why the control has to sit in the instrumentation layer rather than in a guideline.
Where is the real exposure — the provider or our own logs?
Usually your own logs, which is the part that surprises people. Provider terms and retention are at least explicit and negotiable. A trace store is typically readable by a far wider internal audience than the production database the data came from, is often a third-party service, and inherits a retention period nobody chose for this purpose. Both need addressing; the second is where the immediate work is.
What does a workable control look like?
One gateway the calls pass through, where redaction, allowed destinations and retention are enforced in code before anything leaves the process. Per route, choose full text capture, a hash and length, a structural summary naming the template and document ids, or nothing — with full capture in development and structural capture in production for regulated surfaces. Set payload retention short and keep the numeric fields long, since those are what cost and latency analysis actually needs.