Prompt Caching Done Wrong Costs 25% More Than Not Caching At All

Prompt caching gets described as a switch: turn it on, pay less. That framing hides the part that matters, which is that a cache write is not free. It costs more than sending the tokens uncached. So a breakpoint placed where nothing ever reads it does not simply fail to save money — it charges you a premium, on every request, indefinitely.

The mechanism is documented and unambiguous. What is missing from most write-ups is the arithmetic that follows from it, and the prompt-layout discipline that arithmetic implies.

The break-even is one hit

It is worth working this through, because it settles the question of whether caching is worth the complexity for a given endpoint.

Cost of one write + N reads, in multiples of base input price:

    1.25 + 0.10 N

Cost of the same N+1 requests uncached:

    1.00 + 1.00 N

Caching wins when      1.25 + 0.10 N  <  1.00 + 1.00 N
                              0.25    <  0.90 N
                                  N   >  0.28

So a single cache read already repays the write.

  writes only, never read  ->  1.25x per request   (+25% forever)
  1 write, 1 read          ->  1.35x for 2 requests vs 2.00x
  1 write, 9 reads         ->  2.15x for 10 requests vs 10.00x

That result reframes the decision. There is no traffic threshold below which caching is not worth enabling, because one hit covers it. The only question is whether your breakpoint is positioned so that anything ever hits it — and that is a prompt-layout problem, not a volume problem.

Why breakpoints miss

The caching documentation is precise about the mechanism: marking a block with cache_control writes exactly one entry, a hash of the prefix ending at that block. Nothing is written for earlier positions, and a cache hit requires that prefix to match exactly.

So the rule is simple and the common error is equally simple. The breakpoint belongs on the last block whose content is identical across requests. Put it after anything that varies — a timestamp, a user identifier, the current query — and the hash differs every time. Every request writes a fresh entry at 1.25x and no request ever reads one.

# Wrong: the breakpoint sits after per-request content, so the prefix
# hash is unique every call. Every request writes; nothing ever reads.
system = [
    {"type": "text", "text": LARGE_STABLE_INSTRUCTIONS},
    {
        "type": "text",
        "text": f"Current time: {now()}. User: {user_id}.",
        "cache_control": {"type": "ephemeral"},   # <- hash changes each call
    },
]

# Right: breakpoint closes the stable prefix. Volatile content follows it
# and is charged as ordinary input, which is what it should be.
system = [
    {
        "type": "text",
        "text": LARGE_STABLE_INSTRUCTIONS,
        "cache_control": {"type": "ephemeral"},   # <- stable hash, reusable
    },
    {"type": "text", "text": f"Current time: {now()}. User: {user_id}."},
]

The volatile content has not been removed, only moved behind the boundary. It is charged at the normal rate, which is correct — it genuinely is new material on every request. What changes is that the large stable block in front of it is now cacheable.

The hierarchy cascades, and agents break it

The cache prefix follows a strict order: tools, then system, then messages. A change at any level invalidates that level and every level after it. Changing a tool definition therefore invalidates the system prompt and the entire message history along with it.

That has a sharp consequence for agents, which frequently assemble their tool list dynamically — filtering by user permissions, by task, or by which servers responded. If that list is not byte-identical between calls, nothing downstream of it can ever be cached, no matter how carefully the system prompt was structured. This interacts directly with the aggregation behaviour described in when an MCP gateway earns its place: a gateway that returns tools in non-deterministic order will silently destroy caching for every client behind it.

Two failures that produce no error

Both of these return a perfectly normal successful response, which is why caching is so often assumed to be working when it is not.

The minimum cacheable length varies by model — the documentation lists thresholds ranging from 512 tokens on some models up to 4,096 on others. This is a genuine trap when moving between models, because a prompt that cached correctly on one may fall below the threshold on another and silently stop caching, with the only symptom being a cost increase nobody attributes to the model swap.

response = client.messages.create(**request)
usage = response.usage

created = getattr(usage, "cache_creation_input_tokens", 0)
read = getattr(usage, "cache_read_input_tokens", 0)

if created == 0 and read == 0:
    # cache_control was present but nothing cached: almost always a
    # prompt below the model's minimum cacheable length.
    log.warning("caching inactive — check prompt length against model minimum")
elif created > 0 and read == 0:
    # Writing on every request and never reading. This is the 1.25x
    # surcharge case. The breakpoint is on content that varies.
    log.warning("cache written but never read — breakpoint is on volatile content")

Emitting that check as a metric rather than a log line is the version worth shipping. Cache read tokens as a proportion of total input tokens is the single number that tells you whether the feature is working, and it belongs on the same dashboard as spend.

The lifetime clock starts earlier than you think

The default entry lives five minutes, with a one-hour option available at twice the base input price for writes. The detail that catches people is when the clock starts: lifetime is measured from the beginning of the request that writes or reads the entry, not from the end of its response.

Generation time therefore counts against the window. A multi-step agent run where each step takes thirty seconds to generate is consuming its own cache lifetime while it works, and a run long enough can expire its cache mid-execution — reverting to full-price input for the remaining steps, at exactly the point in a long run where the context is largest and the tokens most expensive.

The one-hour TTL is the answer for that shape of workload, and the arithmetic still favours it: a two-times write repaid by reads at a tenth of base price breaks even quickly. It is worth choosing deliberately rather than accepting the default because it was the default.

Common mistakes

1. Placing the breakpoint at the end of the prompt because that is where the most content precedes it. What matters is that the prefix is stable, not that it is long.
2. Assembling tool definitions in non-deterministic order. Tools sit first in the prefix hierarchy, so any variation invalidates everything after them.
3. Putting a timestamp or user identifier in the system prompt before the breakpoint. This is the single most common cause of a write-only cache.
4. Assuming caching is active because cache_control was set. Below the model minimum it is ignored silently, and no error is returned.
5. Reusing a prompt layout across models without rechecking the minimum. Thresholds differ by model, and a working cache can stop working on a model swap.
6. Accepting the five-minute default for long agent runs. The clock starts at request start, so generation time eats the window.
7. Spending the four available breakpoints on fine-grained boundaries. Each one writes an entry at a premium; use them where the content genuinely has different stability lifetimes.

Security and performance considerations

Caches are isolated per workspace on the Claude API, and at organisation level on some cloud platforms. That boundary is worth checking against your tenancy model: if a single workspace serves multiple customers and any customer-specific content sits inside a cached prefix, the isolation guarantee you are relying on is the platform's, not your application's. The clean pattern is to keep the cached prefix entirely free of customer data and place tenant-specific content after the final breakpoint.

On latency, cache reads reduce time to first token materially because the prefix does not need reprocessing, and this is often the larger practical benefit for interactive applications. It is also why cache hit rate belongs in latency dashboards as well as cost dashboards — a drop in hit rate shows up as a user-visible slowdown before anyone notices the invoice.

The failure mode to guard against is the one shared with every other per-call optimisation: individually reasonable requests summing to an unreasonable total. A write-only cache adds 25% to every call and each call still looks fine in isolation, which is exactly the pattern described in why agent bills explode.

Troubleshooting

1. Costs rose after enabling caching — the write-only case. Check whether cache read tokens are consistently zero while creation tokens are not.
2. Both cache fields zero — the prompt is below the model's minimum cacheable length, or cache_control was not actually sent.
3. Hit rate differs between identical deployments — non-deterministic ordering somewhere in the prefix, most often a set or dict iteration order in tool assembly.
4. Caching works for short sessions and fails for long ones — TTL expiry during the run. The clock starts at request start, so generation time counts.
5. Hit rate dropped after a model change — different minimum cacheable length. Recheck the threshold for the new model.
6. First request of the day always slow and expensive — expected. There is nothing to read until something has written, and the write costs more than a plain request.

The useful shift is to stop thinking of prompt layout as a prompt-engineering concern and start treating it as a cost architecture decision. Where you put the boundary between stable and volatile content determines what you are billed on every request, and unlike most cost optimisations it is a change to ordering rather than to content — the prompt says exactly the same thing either way.

If you are running LLM workloads where inference cost is becoming a line item worth managing, that is the kind of work our AI engineering practice takes on. For an existing system where the bill is the presenting problem, a cost review engagement starts by instrumenting what you are actually paying for before recommending changes.