Guardrails

A prompt is a request, not a guarantee. Guardrails are the enforcement layer that does not depend on the model complying: schema validation on outputs, allow-lists on tool calls, PII and secret redaction at the boundary, content filters, spend and rate limits, and human approval gates on irreversible actions.

The design principle is least privilege plus reversibility. An agent should hold the narrowest capability its task needs, and anything hard to undo — money moving, data deleted, messages sent — should require a human. That way a bad output is an annoyance rather than an incident.

Guardrails and evals are complements: evals measure whether quality is good on average, guardrails contain the tail. You need both, because the tail is where the incidents live.