What is LLM inference cost?

Q: What is LLM inference cost?

LLM inference cost is what it costs to actually run a model in production — priced per token on hosted APIs, or per GPU-hour when you host it yourself.

LLM inference cost

Last verified: June 2026· definition

On a hosted API the formula is simple: input tokens × input price + output tokens × output price, where output is typically 3–5x more expensive than input, and reasoning tokens bill as output. Cost therefore scales with usage, which is why AI line items surprise finance in a way fixed infrastructure does not.

Self-hosting flips the shape: you pay for GPUs whether or not requests arrive, so cost per request falls with utilization. That is why self-hosting only pencils out at genuinely high, steady volume — or when data residency makes it the only option.

The number that matters is never the headline price per million tokens; it is cost per successful task, by route. A “cheap” model that needs three retries and a human fix is expensive. That is why cost work starts with instrumentation, not with switching models.