Guardrails
Last verified: June 2026· definition
A prompt is a request, not a guarantee. Guardrails are the enforcement layer that does not depend on the model complying: schema validation on outputs, allow-lists on tool calls, PII and secret redaction at the boundary, content filters, spend and rate limits, and human approval gates on irreversible actions.
The design principle is least privilege plus reversibility. An agent should hold the narrowest capability its task needs, and anything hard to undo — money moving, data deleted, messages sent — should require a human. That way a bad output is an annoyance rather than an incident.
Guardrails and evals are complements: evals measure whether quality is good on average, guardrails contain the tail. You need both, because the tail is where the incidents live.
Guardrails sit outside the model, or they are not guardrails
The distinguishing property is independence from the model deciding to comply. An instruction in a system prompt asking the model not to do something is a request that usually works and sometimes does not. A schema validator that rejects malformed output, an allow-list that refuses an unapproved tool call, or a spend cap that trips regardless of context — those hold even when the model is confused or has been manipulated. If a control can be argued away by cleverly worded input, it belongs in the prompt-quality category, not the guardrail category.
Input guardrails and output guardrails solve different problems
Input-side controls address what reaches the model: redacting secrets and personal data before they leave your boundary, and treating retrieved or user-supplied content as untrusted rather than as instructions. Output-side controls address what leaves the system: schema validation, content filtering, and blocking actions that were not authorised. Systems commonly implement one and assume they have both, which leaves an entire class of failure unaddressed.
Reversibility is the design axis that matters most
The practical question for any agent capability is what happens if it fires incorrectly. Reading data is recoverable. Sending an email, deleting a record, or moving money is not. Sorting capabilities by reversibility, then requiring explicit approval on the irreversible ones, converts the most likely failure mode from an incident into an annoyance — and it does so without needing the model to be more reliable than it is.
Common misconceptions
MythA well-written system prompt is a guardrail.
RealityIt is a request. It raises the probability of good behaviour and provides no enforcement, which is precisely what fails under adversarial input. Guardrails are the controls that still hold when the prompt has been successfully undermined.
MythGuardrails are only needed for user-facing systems.
RealityInternal agents often hold broader permissions than customer-facing ones and read untrusted content — tickets, emails, web pages — that can carry injected instructions. The blast radius is frequently larger internally, not smaller.
MythAdding guardrails degrades the product.
RealityPoorly targeted ones do, by blocking legitimate use. Well-targeted ones are invisible in normal operation because they only engage on the narrow set of actions that are genuinely dangerous, which is a small fraction of what the system does.
Frequently asked questions
What is the difference between guardrails and evals?
They operate at different times and answer different questions. Evals are measurement — they run against datasets or sampled traffic and tell you whether quality is acceptable, before or after deployment. Guardrails are enforcement — they run inline on every request and constrain what the system can emit or do right now. Evals tell you the system is good; guardrails ensure a bad output cannot cause harm. Neither substitutes for the other.
Where should guardrails live in the architecture?
At the boundary the model cannot reach — a gateway, middleware, or the tool layer itself, rather than inside the prompt or in code the agent can influence. Tool-layer enforcement is particularly important for agents: the server exposing a tool should verify identity and permissions itself, because anything passed as a model-supplied argument can be affected by injected instructions.
How do I stop guardrails becoming a bottleneck?
Scope them by risk rather than applying them uniformly. Most calls need only cheap checks like schema validation. Reserve expensive controls — content classification, human approval — for the small set of actions that are irreversible or high-value. Uniform application is what makes guardrails feel slow and drives teams to disable them.
Go deeper
Is your codebase ready for AI agents?
Agents don't fail randomly on a repo — they fail predictably on the same things that slow down human engineers: fuzzy boundaries, missing tests, undocumented conventions. Readiness is just those weaknesses paid down deliberately.
Hiring & partnersHow to choose an MCP server development company
The MCP SDK is a weekend's work; the job you're actually hiring for is least-privilege auth, scoping, audit logging, and production hardening on servers that reach straight into your internal systems.