Self-hosted LLMs vs API models

Run the arithmetic before the architecture

The cost intuition that drives most self-hosting conversations does not survive contact with utilisation. A hosted API bills per token and scales to zero; your own inference stack bills for accelerators whether or not requests arrive. Below a certain sustained utilisation the API is simply cheaper, and the crossover point is a number you can compute from your own traffic profile before committing to anything. Doing that calculation first prevents the most common expensive mistake in this category.

The reasons that do hold up are rarely about price. Data residency or regulatory constraints that make sending content to a third party impossible. A steady, high workload that keeps hardware genuinely busy. A latency requirement that rules out a network hop. A specific open-weight or fine-tuned model no provider hosts. Where one of those applies the economics follow rather than lead — and the ongoing cost people underestimate is not the hardware but that someone now owns an inference platform, with upgrades, incidents and capacity planning attached.

There is also a middle path that gets overlooked. Many teams that assume they must self-host discover that a regional endpoint, a zero-retention arrangement, or a provider's enterprise data terms already satisfy the constraint that started the conversation. That is worth establishing before committing to hardware, because it is a procurement conversation rather than a six-month infrastructure project. Where it genuinely does not suffice, the self-hosted build is the right answer and should be scoped properly rather than half-done.

Where each one genuinely wins

• —Hosted APIs genuinely win on capability, elasticity and operational cost. You get the strongest available models with no capacity planning, no upgrade cycle, and nothing to run — and for spiky or modest workloads the bill is usually lower too.
• —Self-hosting genuinely wins on control and on steady high volume. Data never leaves your environment, there is no per-token meter running, latency has no external hop, and you can run exactly the model you chose for as long as you want it.