Structured outputs
Last verified: June 2026· definition
Structured outputs make a model return data that conforms to a schema you provide, so downstream code can parse it reliably instead of coping with free-form text. This is essential for extraction, classification, and any pipeline step that consumes the model’s output programmatically.
Modern APIs enforce the schema at generation time, eliminating a whole class of parsing errors and brittle prompt hacks (like prefilling a response).
Structured outputs pair with evals and tool use to make LLM features predictable enough for production.
Constrained decoding versus asking nicely
There are two ways to get JSON out of a model and they are not equivalent. Asking for JSON in the prompt produces valid JSON most of the time and fails unpredictably, usually under exactly the unusual inputs where you least want a parse error. Constrained decoding restricts what the model is permitted to emit so that malformed output is not representable at all. The first is a prompt technique with a failure rate; the second is a guarantee, and where a provider offers it there is little reason to rely on the former.
Valid does not mean correct
A schema guarantees shape. It does not guarantee that the extracted value is right, that a required field was not filled with a plausible guess, or that an enum choice was the appropriate one. This distinction matters because schema validation feels like verification and quietly is not — teams frequently add structured outputs, see parse errors disappear, and conclude the extraction is now reliable. Content accuracy still needs evals.
Schema design affects accuracy
The model reads your schema, so field names, descriptions, and enum values act as instructions. Vague names invite guessing; explicit enums outperform free-text fields where the set of valid answers is known; and marking a field optional gives the model somewhere to put uncertainty other than fabrication. Adding a nullable field for cases where information is genuinely absent tends to reduce invented values more than any prompt wording.
It narrows the failure surface rather than removing it
Guaranteeing shape removes parse errors, retry loops around malformed JSON, and a category of defensive code. What remains is semantic: values that are well-formed and wrong, enum choices that are valid and inappropriate, required fields filled with confident guesses. That residue is smaller and much easier to test for, which is the real benefit — you move from handling arbitrary malformed text to validating a known structure.
Not every provider supports it the same way
Support for constrained decoding varies by provider and by model, and the schema subset accepted differs too. Code written against one provider’s strict mode frequently needs adjustment for another, and a schema accepted in one place may be rejected elsewhere. Where portability matters, keeping schemas simple and validating the parsed result yourself gives you a consistent floor regardless of what the provider guarantees.
Common misconceptions
MythStructured outputs stop the model getting things wrong.
RealityThey guarantee the response parses and conforms. Whether the values are accurate is a separate question that only evaluation answers.
MythAsking for JSON in the prompt is the same thing.
RealityIt works most of the time and fails without warning. Constrained decoding makes invalid output impossible rather than unlikely, which is a different guarantee.
MythAny JSON Schema will work.
RealityProviders offering strict modes typically support a subset of the specification, and unsupported constructs are rejected rather than ignored. Schemas often need simplifying to qualify.
Frequently asked questions
Do structured outputs eliminate the need to validate responses?
They eliminate parse errors and schema violations, which removes a whole class of defensive code. They do not tell you whether the content is correct — a confidently wrong value conforms to the schema perfectly. Keep validating semantics: check that identifiers exist, that numbers fall in plausible ranges, and that required relationships hold. What changes is that validation moves from parsing to meaning.
Why is my schema rejected by the provider?
Strict output modes usually support a restricted subset of JSON Schema, and constructs outside it are refused outright rather than quietly ignored. Recursive definitions, some composition keywords, and certain conditional constructs are common causes. The fix is normally flattening the schema — the resulting shape is often easier for the model to fill correctly anyway.
How do I handle fields the model cannot determine?
Give it somewhere legitimate to put that. A nullable field, or an explicit "unknown" enum value, means the model does not have to choose between violating the schema and inventing a value — and faced with that choice it will invent. Making absence expressible is one of the most effective ways to reduce fabricated values in extraction tasks.