Scaling, Quotas & Capacity Planning

Prompt Caching Can Multiply Your Effective TPM Several Times Over

Prompt caching is usually pitched as a cost feature — cache reads bill at a tenth of the input price. The quieter benefit is bigger: on several platforms, cached tokens don't count against your rate limit at all.

Claude 3P 101 · Updated July 2026 · Unofficial guide

Prompt caching lets the platform store a stable prefix of your prompt — the system prompt, tool definitions, a reference document — so repeat requests reuse it instead of reprocessing it. Everyone knows the billing effect: cache reads cost 0.1x the base input price. Fewer teams notice the rate-limit effect, which Anthropic documents explicitly: on the Claude API, cache_read_input_tokens do not count toward your input-tokens-per-minute (ITPM) limit for most models. Uncached input and cache writes do count; reads are free quota-wise. (The one documented exception is the older Haiku 3.5, where reads still count.)

The multiplication

Anthropic's own worked example: an organization with a 2M ITPM limit and an 80% cache hit rate effectively processes 10M total input tokens per minute — only the 20% uncached portion draws down the meter. The general formula is simple:

Effective input throughput = nominal ITPM ÷ (1 − cache hit rate). At 80% hits, 5x. At 90% hits, 10x. The same arithmetic works in reverse for capacity planning: subtract expected cache reads from your token forecast before comparing it to your quota.

For workloads with a large shared prefix — a customer-support bot with a 30,000-token system prompt and knowledge pack, an agent re-sending its tool definitions every turn — this is routinely the single largest throughput lever available, larger than any quota increase you could realistically be granted.

Which platforms give the exemption

Claude API: documented directly in the rate-limits page, as above. Cache-aware ITPM applies to most current models.

Amazon Bedrock (current bedrock-mantle surface): AWS's quota documentation states that cached input tokens read via prompt caching do not count against the mantle input-TPM quota. On the legacy bedrock-runtime side, the CloudWatch metrics reference says the same thing in metric form: CacheReadInputTokens "are charged at a reduced rate and don't count toward your TPM quota," while CacheWriteInputTokens do count.

Microsoft Foundry: rate limits are defined as RPM plus uncached ITPM — the ITPM meter counts uncached input tokens and cache-write tokens, while cache reads (and output tokens) do not count. Same exemption, different wording.

Google Vertex AI: prompt caching itself is fully supported, including on the global endpoint, and cache hits bill at 0.1x input price. But Google's quota documentation does not state that cache reads are excluded from its tokens-per-minute quota metrics. Don't assume the exemption carries over — verify against your project's quota behavior, and treat the cost saving as the guaranteed part.

Claude Platform on AWS: uses the Claude API's rate-limit system, including its cache-aware ITPM accounting.

What still counts

Three fine-print items keep expectations honest. First, cache writes count toward ITPM everywhere they're metered — and bill at a premium (1.25x input price for the 5-minute duration, 2x for the 1-hour). A cold cache, a changed prefix, or a model migration (which invalidates the cache) temporarily removes your multiplier; see the tokenizer-migration article for how those effects stack. Second, cached tokens still occupy the context window — caching changes what you pay and what the rate limiter sees, not how much fits in a request. Third, on the Claude API the exemption applies to ITPM; output tokens are governed separately by OTPM, which caching does not help.

To confirm the multiplier is real in your workload, read the usage object on every response — cache_read_input_tokens versus input_tokens plus cache_creation_input_tokens gives your actual hit rate, and the field layout is consistent across all four platforms. The Claude Console's usage page also charts hourly uncached ITPM against your configured limits.

Where to go next

The mechanics of breakpoints and TTLs live in prompt cache mechanics, and the billing side in cache reads vs writes. For the other big quota levers, see batch offloading and per-model quota pools.

Sources