Tokens
Last verified: June 2026· definition
A token is roughly a word-piece — common words are one token, rarer words and code split into several. Models process and bill by tokens, so token counts, not word counts, drive both cost and context limits.
Different models tokenize the same text differently, so the same prompt can cost a different number of tokens across providers. For accurate budgeting, count tokens with the target model’s tokenizer rather than estimating.
Because both input and output are billed per token (and reasoning tokens count as output on some models), understanding token usage per route is the foundation of any cost-optimization effort.
Why token counts surprise people
Tokens are neither words nor characters. Common English words are frequently a single token, while rare words, technical terms, and identifiers split into several. Code tokenises less efficiently than prose because punctuation, indentation, and camel-case identifiers fragment. Other languages, particularly those not written in Latin script, often consume considerably more tokens per unit of meaning. This is why estimates based on word count are unreliable and why the same text can cost noticeably more in one language than another.
Input and output are priced differently
Output tokens typically cost several times more than input tokens, and on reasoning models the intermediate reasoning bills at output rates even though it never appears in the response. The practical consequence is that a change shortening responses often moves the bill more than one shortening prompts, and that enabling higher reasoning effort has a cost effect well beyond the visible answer length.
Tokens are the unit for limits as well as price
Context windows, rate limits, and throughput quotas are all denominated in tokens rather than requests. A route sending long prompts consumes quota far faster than its request count suggests, which is a frequent cause of rate limiting that appears disproportionate to traffic. Tracking tokens per route alongside requests per route makes this legible.
Estimating before you build
The reliable method is to take a production-shaped prompt — system instructions, a realistic amount of retrieved context, a representative history — and tokenise it, rather than extrapolating from a short test message. Add expected output length, weight it at the output rate, and multiply by projected volume. Estimates built from a bare user message routinely understate real cost by a large multiple, because in production the user message is the smallest component.
Watch tokens per route, not just totals
A total token count tells you what you spent; tokens per route tells you why. Because input size drifts upward as system prompts accumulate instructions and retrieval returns more context, average input tokens per route is the metric that makes cost growth visible while it is happening rather than at the end of a billing cycle. It is also the fastest way to find the route responsible when spend moves unexpectedly.
Common misconceptions
MythA token is roughly a word.
RealityIt is a rough average for ordinary English prose and misleading elsewhere. Code, technical terms, identifiers, and non-Latin scripts all deviate substantially, usually upward.
MythOnly the text you write counts.
RealitySystem instructions, conversation history, tool definitions, retrieved context, and the response all consume tokens. In mature systems these usually exceed the user input by a wide margin.
MythInput and output tokens cost the same.
RealityOutput is typically several times more expensive, and reasoning tokens bill at output rates. Ignoring the asymmetry leads to optimising the cheaper side.
Frequently asked questions
How do I count tokens accurately?
Use the tokeniser matching the model you are calling, since different model families tokenise differently and a count from one does not transfer to another. Providers publish tokeniser libraries for exactly this. Rules of thumb converting words to tokens are adequate for a rough order-of-magnitude estimate and unreliable for budgeting, particularly for code or non-English text where the deviation is largest.
Why does our token usage exceed what we expected?
Usually because the estimate counted only the visible user message. Real requests carry system instructions, accumulated conversation history, tool definitions, and retrieved context, all billed on every call. On reasoning models the intermediate reasoning adds more at output rates. Logging actual input and output token counts per route replaces the estimate with a measurement, and the gap is often large.
Do tokens work the same across languages?
No, and the difference can be significant. Tokenisers are trained on data where English is heavily represented, so English tends to tokenise efficiently while other languages — especially those using non-Latin scripts — often need substantially more tokens to express the same content. For multilingual products this shows up directly as higher cost and lower effective context for some users than others.
Go deeper
How to reduce LLM API costs
Most production LLM spend is avoidable: it comes from sending the wrong model too many tokens, too many times. Cutting it is an engineering problem — routing, caching, batching, and context control — not a matter of waiting for prices to drop.
LLM costWhy are LLMs so expensive?
LLMs are expensive because you pay per token on every call, output usually costs more than input, and agents re-read a growing context on every step — so cost compounds far faster than the work appears to grow.
LLM costHow to reduce LLM latency
LLM latency is two separate problems — the wait for the first token and the time to stream the rest — and each has its own set of fixes.
LLM evalsLLM observability: what to instrument in production
LLM observability is the practice of recording enough about every model call that you can answer, after the fact, what it cost, how long it took, whether the output was any good, and what the model was actually shown. Standard APM answers none of those, because a call that returns HTTP 200 in 900ms can still be a complete failure.