Prompt compression

Because input tokens are billed on every single call, a bloated system prompt is a tax you pay forever. Teams accumulate instructions defensively — a rule added for one edge case in March, three redundant restatements of the same constraint, a dozen few-shot examples where two would do.

The practical moves: consolidate overlapping instructions, drop examples that no longer earn their place, inject few-shot examples conditionally rather than always, retrieve fewer and better chunks instead of stuffing context, and ask for structured output so the model stops narrating.

Compression pairs with caching rather than competing with it: cache the stable prefix you cannot remove, and delete the parts you never needed. Both belong behind an eval, since a prompt is a program and deleting lines can change behavior.