Context window

The context window bounds how much the model can “see” at once — the system prompt, conversation history, retrieved documents, and the response all draw from it. Exceeding it forces truncation, summarization, or compaction.

Larger context windows (now up to ~1M tokens on frontier models) enable whole-codebase reasoning and long documents, but using more context costs more tokens and can dilute focus, so it’s not always better.

Managing the context window well — via retrieval, compaction, and caching — is central to building cost-effective, reliable LLM systems.