Amazon Bedrock publishes EstimatedTPMQuotaUsage in the AWS/Bedrock CloudWatch namespace to approximate how much of your tokens-per-minute (TPM) quota a model is consuming. It is a genuinely useful trend line. But AWS attaches an explicit caveat: the metric does not reflect the reservation-based accounting that actually decides whether your next request gets throttled, and AWS says not to use it as your sole capacity-planning signal. Teams that alarm on "EstimatedTPMQuotaUsage < 80%" and call capacity planning done are measuring the wrong thing.
Reservation accounting: why the estimate diverges
When a request arrives, Bedrock does not wait to see how many tokens the response actually uses. It reserves capacity upfront: your input tokens plus the full max_tokens value you asked for are charged against the TPM quota at admission time. If Claude then answers in 200 tokens against a max_tokens of 8,000, the unused reservation is eventually returned — but during the request, your quota headroom was reduced by the full reservation.
This is why the estimate and reality diverge. EstimatedTPMQuotaUsage approximates consumption, while throttling decisions run on reservations. A fleet of requests with generous max_tokens and short actual outputs can be throttled hard while the estimate reports comfortable headroom. (The current bedrock-mantle surface documents the same admission-control model — input tokens plus max_tokens reserved against input TPM, 429 on exceed, with unused reservation replenished after completion — so the lesson carries across surfaces; see mantle admission control.)
What to watch instead: throttles, with eyes open
The ground-truth signal is InvocationThrottles — the count of requests Bedrock actually rejected for capacity reasons. Two documented subtleties make it trickier than it looks:
- Throttled requests are invisible elsewhere. A throttled request counts as neither an
Invocationsdata point nor a client/server error. If you compute a throttle rate, the denominator isInvocations + InvocationThrottles, notInvocationsalone. - SDK retries inflate (or hide) the count. The AWS SDKs retry throttled requests by default, and each retried attempt that gets throttled increments the metric again. The same user-facing workload produces very different
InvocationThrottlescounts under different retry configurations — compare numbers only across identical retry settings, and remember that a "low" throttle count with aggressive retries may just mean your users are absorbing latency instead of errors.
Round out the picture with InputTokenCount / OutputTokenCount for real consumption trends, and the cache metrics: CacheReadInputTokens are billed at a reduced rate and — importantly for capacity — do not count toward your TPM quota, while CacheWriteInputTokens do. A workload with a high cache hit rate has materially more effective quota than its raw token volume suggests.
Sizing max_tokens conservatively
Because reservations run on max_tokens, the single cheapest capacity lever is not a quota increase — it is honest max_tokens values. Practical guidance:
- Size per use case, not per application. A classification endpoint that never emits more than 50 tokens should not share the 8,000-token
max_tokensits sibling summarizer needs. Every excess token in the parameter is quota reserved for output that never comes. - Measure actual output distributions from your logs or the
usageobject, then setmax_tokensnear a high percentile with modest headroom — enough to avoid truncation, not "the model maximum, to be safe." - Revisit after model or prompt changes. Output length distributions shift; a value tuned last quarter may now be double what the workload needs.
EstimatedTPMQuotaUsage, alarm on InvocationThrottles, and fix with max_tokens hygiene before requesting quota increases. If throttles persist with honest reservations, that is your genuine signal to pursue a quota increase.Where to go next
Throttling handling on Bedrock covers the client-side retry story, and TPM reservation on the mantle surface goes deeper on the current endpoint's admission control.