Cost Optimization & FinOps

Maximizing Cache Hit Rates: Prompt Structure and TTL

Prompt caching sells input tokens at a 90% discount — but only when the bytes match exactly. Hit rate is an engineering property of how you author prompts, not luck.

Claude 3P 101 · Updated July 2026 · Unofficial guide

Prompt caching lets Claude reuse the processed form of a prompt prefix across requests. The economics are simple: writing to the cache costs 1.25x the base input price (5-minute TTL) or 2x (1-hour TTL), and every subsequent read costs 0.1x. On Claude Opus 4.8, that is $6.25 or $10 per million tokens to write and $0.50 to read, against $5 uncached. The 5-minute cache pays for itself after a single read; the 1-hour cache after two. Everything else in this article is about making those reads actually happen.

The cache key is exact prefix bytes

Caching is a strict prefix match: the key is the exact bytes of the rendered prompt up to each breakpoint. One byte changed at position N invalidates every cached segment at or after N. There is no fuzzy matching, no "close enough." This single fact explains almost every disappointing hit rate, and it dictates the authoring habits:

Nothing volatile in the shared prefix. The documented silent invalidators are timestamps or UUIDs in the system prompt, non-deterministic JSON serialization (missing sort_keys=True in Python), per-user tool sets, and conditional system-prompt sections. If the model genuinely needs the current date, put it in the final user message, after the cached prefix.

Respect the hierarchy. The cache is ordered toolssystemmessages, and a change at any level invalidates that level and everything after it. Editing one tool description invalidates all caches; changing the system prompt invalidates system and messages; only message-level changes are contained. Treat tool definitions as the most change-sensitive bytes you own and version them deliberately. Note the subtler invalidators too: toggling web search or citations invalidates system and messages, while changing tool_choice or adding and removing images breaks only the message cache.

Stable prefix, variable suffix. Order every request as: tool definitions, system prompt, shared examples and documents, then per-request content last. This is the same discipline as template design for prompt compression — the two optimizations reinforce each other.

Placement and mechanics

Two ways to enable caching: automatic caching via a single top-level cache_control field (the recommended default — the API places the breakpoint on the last cacheable block and moves it forward as the conversation grows), or explicit cache_control: {"type": "ephemeral"} breakpoints on individual content blocks, up to 4 per request. Explicit breakpoints matter for multi-tier prefixes — for example one breakpoint after the never-changing tool definitions and another after a per-tenant document. Watch the lookback rule in agentic loops: each breakpoint looks back at most 20 content blocks for a prior cache entry, so long turns that append more than 20 blocks can silently miss; place an intermediate breakpoint roughly every 15 blocks.

Minimums and scope: the minimum cacheable prompt is 1,024 tokens on Opus 4.8, Sonnet 5, Sonnet 4.6, and Haiku 4.5, and 512 tokens on Fable 5 (1,024 on Bedrock). Caches are model-scoped — switching model strings forces a full rebuild — and platform support differs: automatic caching is unavailable on Amazon Bedrock (explicit breakpoints only), and cache isolation is per workspace on the Claude API, Claude Platform on AWS, and Foundry, but per organization on Bedrock and Google Cloud.

Choosing the TTL

TTLWrite costBreaks even atFits
5 minutes (default)1.25x input price2 requestsInteractive sessions, steady traffic that re-hits within minutes
1 hour (ttl: "1h")2x input price3 requestsSparse traffic, long agent runs, batch jobs (batches can exceed 5 minutes)

Reads also refresh the entry, so steady traffic keeps a 5-minute cache alive indefinitely; the 1-hour TTL is for gaps, not volume. For high fan-out jobs, remember a cache entry becomes readable only once the first response begins streaming — warm it first (a max_tokens: 0 pre-warm request is supported, with documented exclusions), then release the parallel requests.

Measure it

Every response reports cache_creation_input_tokens, cache_read_input_tokens, and input_tokens (tokens after the last breakpoint). Your hit rate is read tokens divided by the sum of all three; a falling ratio after a deploy almost always means someone touched the prefix. One caution: cached tokens still occupy the context window — caching changes what you pay, not what fits.

Where to go next

See prompt cache mechanics, system prompt caching ROI, and cache write vs. read economics.

Sources