Prompt caching: the AI cost control most teams don't measure

Coding agents send the same large prefix with almost every request: system instructions, tool definitions, project files and the conversation so far. Model providers charge much less for input they have already seen, through prompt caching. Whether an agent hits that cache can change its input bill several times over, yet few teams look at cache hit rates at all. This article explains how caching is priced, why agents depend on it, and what to measure.

How prompt caching is priced

Both major API providers cache the beginning of a prompt and charge less when a later request starts with the same prefix.

The Claude Platform documentation lists the multipliers: writing to a 5-minute cache costs 1.25 times the base input price, writing to a 1-hour cache costs 2 times, and reading from the cache costs 0.1 times, with lower read multipliers for some newer models. There is a minimum cacheable length that varies by model.

The OpenAI documentation describes a minimum cacheable prefix of 1,024 tokens for GPT-5.6 and later, and lists a cache read charge of 0.1 times the uncached input price for those models, with model-dependent pricing for earlier ones. Responses report cached_tokens so you can see how much of each request was served from the cache.

The pattern is the same across providers: a cache read costs roughly a tenth of a normal input token, and a cache write costs somewhat more than a normal one.

Why agents depend on it

A coding agent's conversation grows with every step. Each request sends the system prompt again, the tool definitions and everything that came before, plus a little new content. If the prefix stays identical, most of each request is a cheap cache read. If anything near the start changes, the whole prefix after it is a new write, at more than full price.

Things that silently break the prefix include:

• tool definitions returned in a different order, or changed mid-session;
• timestamps or other changing values in the system prompt;
• switching models mid-conversation, since caches are per model;
• long pauses that outlast the cache's lifetime.

The new MCP specification addresses the first of these. The MCP release post says list responses now carry cache hints "and a deterministic order, so clients can cache tool catalogs and keep upstream prompt caches stable across reconnects."

Analysis: what to make of it

It is a large lever. With cache reads at a tenth of the input price, the difference between a well-cached and a poorly cached agent run can be bigger than the difference between two models. Teams that compare models on list price alone miss this.

Writes are not free. At 1.25 or 2 times the base price, writing a cache that is never read costs more than not caching. A long cache lifetime pays off only if the same prefix is reused within it.

Model routing interacts with caching. Sending each call to the cheapest adequate model sounds efficient, but switching models mid-conversation discards the cache. Routing decisions are better made per conversation than per call.

You can measure it. Providers report cache reads and writes in every response, and coding tools pass them on. Claude Code's telemetry, for example, splits its token counter into input, output, cache read and cache creation.

What it means for engineering and security leaders

If you are trying to control AI spending, cache behaviour belongs on the same dashboard as model choice and per-person budgets. A sudden fall in cache hit rate after a tool update, a new MCP server or a changed system prompt is a cost incident, even if nothing else changed.

It is also a reason to be careful with changes that look harmless: adding a timestamp to an agent's instructions, or a plug-in that reorders tools, can raise costs across a whole team.

Practical takeaways for this week

• Add cache read and cache write tokens to your AI cost reports, next to input and output tokens.
• Compute a simple ratio per tool and per team: cache reads divided by all input tokens. Watch it over time.
• Keep the start of prompts stable: no timestamps or per-request values in system instructions; put changing content at the end.
• Keep tool lists stable within a session, and prefer MCP servers that return them in a fixed order.
• Make model-routing decisions per conversation, not per call, so caches survive.
• Before choosing a longer cache lifetime, check that the same prefix is reused often enough to pay for the higher write price.

Where SpanDock fits

With the SpanDock model proxy, in every plan, a conversation keeps its model for 6 hours, so the model doesn't change mid-task, and every call records the requested and answering model, tokens and cost. The Models page ranks the models your tools used by spend or cost per million tokens.