How to hire an LLM cost optimization consultant

Do you even need a consultant?

Start with the honest question, because the honest answer is sometimes no. LLM cost optimisation is not deep magic — it is routing, caching, batching, context control, and the evals that keep quality from slipping while you do it. A strong platform or ML team can do all of it. The reason teams bring in help is rarely capability; it is capacity and reps. The people who could fix the bill are the same people shipping the roadmap, and cost work keeps losing the priority fight until the bill is embarrassing.

The other real reason is pattern recognition. A specialist who has instrumented a dozen LLM workloads knows where the spend usually hides — the agent loop that re-reads its context every step, the prompt that ships a whole document when it needed two paragraphs, the frontier model quietly serving a classification task — and can get to the root cause in days rather than the weeks it takes a team seeing it for the first time.

But the bill has to justify it. If you are spending a few hundred a month, an outside engagement will cost more than it saves and you should read a reduction guide and spend an afternoon on it. The economics start to work when spend is large enough that a 30–50% reduction is real money, when it is growing month over month with no one accountable for it, or when the workload is agentic and scaling super-linearly. If your bill is five or six figures a month and trending up, the question is not whether to invest engineering time — it is whose.

In-house vs hire: how to actually decide

Frame it as a build-versus-buy on engineering time, not on the consultant's fee in isolation. The in-house cost is not zero; it is the loaded cost of your best platform engineer, plus the opportunity cost of what they are not shipping, plus the calendar time to climb a learning curve someone else has already climbed. Against that, weigh a fixed engagement that returns a lower bill and, ideally, the instrumentation to keep it low.

Keep it in-house when you have genuine slack on a capable team, when the workload is simple and single-model (routing has little to bite on), or when the domain is so sensitive that onboarding an outsider costs more than the saving. Bring someone in when the team is at capacity, when the workload is complex or agentic, when the bill is urgent enough that the multi-week in-house ramp is itself expensive, or when you want an independent, vendor-neutral read on where the money is going before you commit engineers to it.

There is a sensible hybrid that often wins: hire a specialist for a short, sharp engagement to instrument the system, find the hotspots, and build the first routing-and-caching pass with the eval harness around it — then have your team own and extend it. You buy the reps and the fast start; you keep the capability. That is usually the highest-return version of this decision, and a good consultant will actively steer you toward it rather than toward a dependency on them.

What a good LLM cost consultant actually does

The work follows a predictable arc, and you should recognise it in any proposal. First, audit and instrument: capture tokens in and out, cost, latency, and a quality signal per route — per prompt template, per model, per feature — because you cannot cut what you cannot see. Most teams are flying blind here, and this step alone often surfaces the surprise: a handful of routes usually dominate the bill.

Then they attack the hotspots in order of leverage. Model routing comes first: send each request to the cheapest model that still clears the quality bar for that task, with the strong model as a fallback. Then caching and batching: prompt-cache stable prefixes, add a semantic cache for repeat-shaped questions, and move non-urgent work (backfills, evals, bulk enrichment) onto batch APIs that price well below real-time. Then input and output control: retrieve only the context a prompt needs, prune agent history so loops stop going quadratic, and constrain output tokens, which are the expensive ones.

The part that separates a real practitioner from a cost-cutter is that every one of those changes is gated on evals. You only downgrade a route once a regression suite proves quality holds on that specific prompt. Cutting cost without an eval harness is just degrading your product slowly, and the damage shows up in churn long after the invoice looks better. Finally comes the handoff: cost-and-quality dashboards wired into your stack, the eval suite in your repo, and a short runbook so the savings survive after they leave.

How to evaluate one

Judge candidates on four things, and treat all four as non-negotiable. First, evidence they measure. Ask how they would establish a baseline and prove a saving. A serious answer talks about per-route instrumentation, cost per successful task, and before/after comparison on the same eval set. A weak answer talks in percentages with no method behind them. Anyone quoting you a guaranteed number before they have seen your traffic is guessing.

Second, vendor neutrality. A good consultant is indifferent to which model or gateway you end up on and picks per route on the numbers. Be wary of anyone whose entire recommendation routes through a product they resell or a provider they are partnered with — their incentive and yours have quietly diverged. Third, quality discipline. If evals are not central to how they describe the work, they are optimising the invoice and not the product; walk away.

Fourth, and most important, they leave you owning it. The deliverable should be changes in your codebase, dashboards in your observability stack, and evals in your repo — not a hosted black box that only works while you keep paying them. A consultant confident in their work is happy to hand it over and be judged on whether the saving sticks. One who structures every engagement so you cannot maintain the result without them is selling dependency, not optimisation.

Engagement and pricing models

There are three common shapes, and the right one depends on how well-defined the work is. Fixed-scope is the most common and usually the cleanest: a defined engagement — audit, then build routing, caching, and evals for an agreed set of workloads — for a fixed fee and timeline. You know what you are paying and what you get, and the incentive is to deliver and hand off. This suits most first engagements because the scope is knowable once the audit is done.

Outcome- or success-based ties the fee to the saving delivered — for example, a share of the measured reduction over a baseline. It aligns incentives neatly and de-risks the spend for you, but only works when both sides trust the measurement, so it depends entirely on a clean, agreed baseline and eval set. Watch for the failure mode where cost is cut by quietly degrading quality; the eval gate is what keeps success-based honest, which is why a serious practitioner will insist on it before agreeing to this model. Retainer or fractional — a fixed slice of someone's time each month — fits ongoing tuning as your workload and the model landscape keep shifting, or a team that wants a standing expert without a full hire. It is the right model for continuous work and the wrong one for a one-off fix, where it can drift into paying for presence rather than outcomes.

Whatever the model, expect the audit to be a distinct, low-commitment first step. A short, read-only diagnostic that quantifies the opportunity before you commit to a build is a good sign — it means the consultant is willing to show you the number before you sign for the work, and it gives you a real basis to decide whether the larger engagement is worth it. Be sceptical of anyone who wants a large open-ended commitment before they have looked at a single trace.

Questions to ask and red flags

A few questions do most of the filtering. "How will you establish the baseline and prove the saving?" — you want a measurement method, not a percentage. "How do you make sure quality does not slip?" — you want evals at the centre of the answer. "What do we own when you leave, and can we maintain it without you?" — you want code, dashboards, and evals in your stack. "Are you tied to any model, gateway, or tool you would recommend?" — you want honesty about incentives. "What does the first two weeks look like?" — you want audit-and-instrument, not an immediate rebuild.

The red flags are consistent. Magic-model claims: "just switch everything to model X and save 70%" ignores that the cheap model fails on your hard prompts and that per-token price is not cost per finished task. No evals: any plan to cut cost without a quality gate is a plan to degrade your product. Lock-in: a black-box gateway or dashboard you rent forever rather than own. No measurement: savings quoted with no baseline and no method to verify them. And guaranteed numbers up front: real savings depend on your actual traffic, which they have not seen yet.

One subtler flag: a consultant who never says no. Sometimes the right answer is that your bill is too small to justify the work, or that your team should do this themselves with a day of guidance. Someone who tells you that when it is true is someone worth trusting with the engagements where hiring genuinely is the right call.

How an engagement runs end to end

A well-run engagement is short and legible. It opens with a read-only audit: they instrument your workloads and produce a ranked picture of where the money goes and an honest estimate of the recoverable amount. If the number is small, this is where a good consultant tells you so. If it is worth pursuing, you move to a fixed-scope build against the hotspots the audit found — routing the biggest routes to cheaper models behind eval gates, adding caching and batching where the traffic supports it, and trimming input and output on the worst offenders.

Throughout, changes land in your repository and behind your review, not on a parallel system you cannot see. Each optimisation is validated against the eval suite before it ships, so you can watch cost fall on the dashboard while quality holds on the same chart. The engagement closes with a handoff: the eval harness and dashboards are yours, there is a short runbook for extending the work to new routes, and your team can carry it forward. From there you either take it fully in-house or keep a light fractional arrangement for ongoing tuning — a deliberate choice you make with the numbers in front of you, not a dependency you backed into.