Batch inference (Batch API)

Most major providers offer a batch endpoint at roughly half the standard per-token price, with results returned within a window (commonly up to 24 hours) rather than immediately. You trade latency you do not need for money you would rather keep.

The fit is anything not sitting in front of a waiting user: nightly enrichment, backfills, generating embeddings over a corpus, running an eval suite, bulk classification, offline summarization. In most production systems a surprising share of total token volume is exactly this kind of work, still being paid for at interactive prices.

The engineering cost is real but modest: you need a job queue, result reconciliation keyed by request ID, and a retry path. It is usually the least controversial cost lever available, because nothing user-facing changes.