Bedrock enforces per-model quotas on how fast your account can consume Claude — requests and tokens per minute (see the quota dimensions guide). When you exceed them, requests are rejected rather than queued. On the current "Claude in Amazon Bedrock" surface this arrives as an HTTP 429: the endpoint's admission control reserves your input tokens plus max_tokens against the input-TPM quota and rejects the request if the reservation won't fit. On the legacy surface, AWS SDKs surface rate rejections as ThrottlingException, with ServiceQuotaExceededException raised for quota-level breaches.
Two errors, two meanings
Throttling errors are transient. They mean "too fast right now" — a burst momentarily exceeded your per-minute allowance. The correct response is to wait briefly and retry; on the mantle endpoint, unused reservation from completed requests is replenished, so pressure genuinely eases within the minute.
Quota-exceeded errors are structural. A sustained pattern of them means your steady-state demand exceeds your allocation. Retrying harder cannot fix arithmetic: you need to reduce token consumption, spread load, or request a quota increase. Treat a rising rate of these errors as a capacity-planning alarm, not a reliability bug.
Log the two separately. A dashboard that lumps them together tells an on-call engineer nothing about whether to wait or to escalate.
Exponential backoff with jitter, done right
Exponential backoff means doubling your wait between successive retries (1s, 2s, 4s, 8s…) so a congested account isn't hammered at a fixed rhythm. Jitter means randomizing each wait, so a fleet of workers throttled at the same moment doesn't retry in lockstep and re-throttle itself — the "thundering herd" problem. Both parts are mandatory; backoff without jitter merely synchronizes the herd on a slower beat.
import random, time
from anthropic import AnthropicBedrockMantle
client = AnthropicBedrockMantle(aws_region="us-east-1")
def create_with_backoff(max_retries=6, **kwargs):
for attempt in range(max_retries):
try:
return client.messages.create(**kwargs)
except Exception as err:
if not is_throttle(err) or attempt == max_retries - 1:
raise
delay = min(random.uniform(0, 2 ** attempt), 30)
time.sleep(delay)Implementation notes: is_throttle should return true only for rate-limit errors (HTTP 429 / throttling exception types from your SDK) — never blind-retry validation or access errors. Cap both the retry count and the maximum sleep. The version above uses "full jitter" (a random delay between zero and the exponential ceiling), a standard pattern that spreads retries evenly. Also check what your SDK already does: AWS SDKs and the anthropic SDK ship built-in retry behavior, and stacking an aggressive custom loop on top of built-in retries multiplies your effective attempt count.
Reduce the throttles you generate
The best retry is the one you never need. Three levers reduce throttle pressure on Bedrock specifically:
Right-size max_tokens. On the mantle endpoint, admission control reserves your full max_tokens up front. Requesting the model maximum "just in case" makes each in-flight call occupy far more quota than it will use.
Use prompt caching. Cached input tokens read via prompt caching do not count against the mantle input-TPM quota — a large shared system prompt or document prefix, cached, stops consuming your rate limit on every call.
Spread the load. Cross-region inference profiles route traffic across regions and draw on cross-region quota pools; see the failover design guide for using them deliberately.
max_tokens, and quota planning — not more retries.Don't confuse throttling with timeouts
One adjacent failure mode masquerades as flakiness: long generations. The inference timeout for recent Claude models on Bedrock is 60 minutes, but AWS SDK clients default to a 1-minute read timeout — AWS recommends raising it (for example, botocore read_timeout of 3600 or more). A client-side timeout is not a throttle; retrying it re-runs an expensive request that was proceeding fine. Classify before you retry.
Where to go next
Size your real limits with Bedrock quota dimensions, and see common Bedrock errors for the failures that should never be retried.