How to fix: request timeout on large or streaming responses

Last verified: June 2026· Long generations and high max_tokens

Where this shows up

Long generations and high max_tokens

The fix

1Switch to streaming for any request that may produce long output (e.g. max_tokens above ~16K).
2Use the SDK’s stream helper and collect the final message rather than awaiting one big response.
3Confirm your client timeout units — some SDKs use milliseconds, others seconds — and raise the timeout if needed.
4For very long agentic turns, stream progress to the user so the connection stays active and the UX reflects work in progress.

Prevent it

Default to streaming for anything with large output, and design timeouts and progress UX around minutes-long turns on frontier models.

Common variations and related errors

You'll usually hit this same root cause under a few different names. Same fix.

"request timeout"
"ETIMEDOUT"
"upstream timeout"
"gateway timeout (504)"
"context deadline exceeded"
fetch failed: timeout

Code: drop-in retry helper

A minimal typescript helper that handles the most common version of this error. Copy, paste, adjust the policy to your traffic.

// Stream long generations instead of waiting for the full response.
import OpenAI from 'openai';
const client = new OpenAI();

const stream = await client.responses.create({
  model: 'gpt-4o',
  input: prompt,
  max_output_tokens: 16000,
  stream: true,
});
for await (const event of stream) {
  if (event.type === 'response.output_text.delta') {
    process.stdout.write(event.delta);
  }
}

Frequently asked questions

What causes “request timeout on large or streaming responses”?

A non-streaming request with a high `max_tokens` exceeds the SDK/HTTP timeout before the full response returns.

How do I prevent “request timeout on large or streaming responses” from recurring?

Default to streaming for anything with large output, and design timeouts and progress UX around minutes-long turns on frontier models.