Streaming, Errors & Resilience

ITPM and Cache Reads: Why a High Cache Hit Rate Multiplies Your Effective Throughput

Prompt caching is usually sold as a cost optimization. On the Claude API it is also a capacity optimization: tokens served from cache mostly don't count against your input rate limit, so the same quota can carry several times the traffic.

Claude 3P 101 · Updated July 2026 · Unofficial guide

Every Claude API organization has, per model, an input-tokens-per-minute (ITPM) rate limit — one of the three ceilings (alongside requests and output tokens per minute) that determine how much traffic you can push. Teams doing capacity planning usually compute "total input tokens per request × requests per minute" and compare it to their ITPM number. With prompt caching in the picture, that arithmetic overstates your consumption, sometimes dramatically, because of a documented accounting rule: for most Claude models, input_tokens and cache_creation_input_tokens count toward ITPM, but cache_read_input_tokens do not.

The three buckets of input

Prompt caching lets you mark a stable prefix of your prompt — system instructions, tool definitions, a big reference document — with cache_control so subsequent requests reuse it instead of reprocessing it. Once caching is active, the usage object splits input into three fields, and only two of them draw down your rate limit:

Usage fieldWhat it isCounts toward ITPM?
input_tokensUncached input after the last cache breakpointYes
cache_creation_input_tokensTokens being written into the cacheYes
cache_read_input_tokensTokens served from an existing cache entryNo (most models)

Anthropic's own worked example shows the leverage: an organization with a 2M ITPM limit that achieves an 80% cache hit rate can effectively process 10 million total input tokens per minute — five times the nominal quota — because only the uncached 20% is metered against the limit. The cached tokens still appear in your usage data and your bill (at a heavily discounted 0.1× rate), but they don't consume rate-limit capacity.

The one documented exception

The carve-out does not apply everywhere: on Claude Haiku 3.5, cache reads do count toward ITPM. If you run that older model, compute capacity the pessimistic way. For current-generation models (the Opus 4.x line, Sonnet 5, Haiku 4.5, Fable 5), the exclusion applies as described. As always, per-model limits differ — Fable 5, for instance, carries lower ITPM numbers at each usage tier than the Opus and Sonnet classes — so check your actual quota per model in the Console.

What this means for architecture, not just billing

Cache hit rate becomes a throughput lever. If you're rate-limit constrained, restructuring prompts so more of each request is a stable cached prefix can be a bigger unlock than a tier upgrade. The usual caching hygiene applies with extra force: caching is a strict prefix match, so timestamps, UUIDs, or non-deterministic JSON serialization in the shared prefix silently destroy hits — and with them, your effective throughput multiplier.

Watch the write column. Cache writes still count toward ITPM (and cost 1.25× or 2× base input price depending on TTL). A workload that churns its cache — rotating prompts, per-user tool sets, frequent invalidations — pays rate limit and money for writes without accumulating read benefits.

Instrument the split. Because effective capacity now depends on hit rate, monitor the three usage fields separately. The Console's Usage page reports hourly max uncached ITPM alongside cache rates, which is exactly the pair of numbers this accounting makes decision-relevant.

Rule of thumb: effective input capacity ≈ ITPM ÷ (your uncached fraction). At a 50% hit rate a 2M quota behaves like 4M; at 80% like 10M. Measure your real hit rate before buying more quota.

The Bedrock parallel

Interestingly, the same principle is documented on Amazon Bedrock independently: on the current bedrock-mantle surface, cached input tokens read via prompt caching do not count against the input-TPM quota, and AWS's CloudWatch metrics annotate CacheReadInputTokens as not counting toward the TPM quota (while CacheWriteInputTokens do). Vertex AI and Foundry meter through their own quota systems — don't assume the exclusion there without checking those platforms' documentation.

Where to go next

Caching fundamentals live in prompt cache mechanics and the read/write economics in cache reads vs. writes. For the scaling playbook built on this rule, see prompt caching as effective throughput; for the Bedrock-side version, Bedrock's cache-read quota exemption.

Sources