How to fix: model returned invalid / unparseable JSON

How it shows up

The same root cause surfaces in a few different ways. If any of these match, you're in the right place.

• ▸A parse error on output that looks correct at a glance but carries a prose preamble such as "Here is the JSON you asked for".
• ▸Valid JSON that fails schema validation because a field is missing, renamed, or returned as a string where a number was expected.
• ▸Output wrapped in a markdown code fence, so the first character the parser sees is a backtick.
• ▸Intermittent failures on long responses, where the JSON is well formed up to the point it was cut off.

Diagnose it first

Confirm which cause you actually have before changing anything — most wasted time here comes from fixing the wrong one.

Provider-specific behaviour

The error means roughly the same thing everywhere, but each provider signals it differently.

Structured output versus JSON mode

These are not the same guarantee. A basic JSON mode typically promises syntactically valid JSON and nothing about the fields; a schema-constrained mode enforces the shape during decoding. Check which one the endpoint you are calling actually provides.

Schema complexity

Deeply nested schemas, unions and open-ended maps are handled less reliably than flat objects with explicit fields, and some providers reject constructs their constrained decoder cannot express. Flattening the schema is often faster than fighting it.

What doesn't work

Common fixes for this error that look right and aren't.

• Stripping markdown fences with a regular expression and moving on.

  It works until the model returns a fence inside a string value, and it leaves the underlying cause — that you are asking rather than constraining — in place. Use the provider's structured output mode instead.

• Adding 'ONLY return valid JSON' in capitals to the prompt.

  Emphasis reduces the frequency of preambles without eliminating them, so you end up with a rare failure that surfaces in production rather than in testing. A constrained decoder removes the class of failure.

• Retrying the same request on a parse failure.

  With unchanged input and a stable model this often produces the same shape of output, and it doubles the cost. Feed the parse error back as a repair turn, or fix the constraint.

Common variations and related errors

You'll usually hit this same root cause under a few different names. Same fix.

• ▸"Invalid JSON"
• ▸"JSON parse error"
• ▸"expected property name"
• ▸"model returned malformed JSON"
• ▸"function call arguments invalid"
• ▸openai.InvalidJSONOutput
• ▸"tool_use_failed"

Code: drop-in retry helper

A minimal typescript helper that handles the most common version of this error. Copy, paste, adjust the policy to your traffic.

// Always request structured outputs (schema-enforced) for JSON routes.
// If you can't, retry with a clarifying instruction and surface the error.
import OpenAI from 'openai';
const client = new OpenAI();

async function extractOrder(text: string) {
  const schema = {
    type: 'object',
    properties: { orderId: { type: 'string' }, total: { type: 'number' } },
    required: ['orderId', 'total'],
    additionalProperties: false,
  } as const;
  const res = await client.responses.create({
    model: 'gpt-4o-mini',
    input: text,
    text: { format: { type: 'json_schema', name: 'order', schema, strict: true } },
  });
  return JSON.parse(res.output_text);
}