Handling Rate Limits and Throttling on Foundry

Rate limiting is how any API protects shared capacity: exceed your allowance and requests are rejected with HTTP 429 ("Too Many Requests") until the window resets. On Microsoft Foundry, Claude rate limits have a specific shape, a subscription-level scope, and one significant gap compared with the first-party API — Foundry does not return Anthropic's anthropic-ratelimit-* response headers. Anthropic's own guidance for Foundry is blunt: manage rate limiting through Azure monitoring tools and exponential backoff on 429s.

How Foundry measures your usage

Two meters, per Microsoft's documentation: RPM (requests per minute) and ITPM (uncached input tokens per minute). ITPM counts uncached input tokens plus 5-minute and 1-hour cache-write tokens; output tokens and cache reads do not count. Two consequences follow. First, prompt caching directly relieves rate-limit pressure, not just cost. Second, long outputs are "free" from a throttling perspective — a workload generating huge responses from small prompts is limited mainly by RPM.

Scope matters as much as the numbers: quota is managed at the subscription level. All Global Standard deployments of the same model and version in a subscription draw from one shared pool across regions (Data Zone Standard deployments share a pool per data zone). A batch job hammering claude-sonnet-5 in one resource can throttle a customer-facing app using the same model in another resource under the same subscription.

Yes, that Fable 5 row means what it says: on pay-as-you-go subscriptions the default quota is zero — you must obtain quota before the first successful request. Increases beyond defaults go through Microsoft's quota increase request form.

Backoff that behaves in production

Exponential backoff means waiting longer after each consecutive failure — 1s, 2s, 4s, 8s — with random "jitter" added so a fleet of workers doesn't retry in lockstep. Since Foundry gives you no header telling you when the window resets, jittered exponential backoff with a retry cap is the whole strategy:

import random, time
from anthropic import AnthropicFoundry, RateLimitError

client = AnthropicFoundry(api_key="...", resource="example-resource")

def call_with_backoff(retries=5, **kwargs):
    for attempt in range(retries):
        try:
            return client.messages.create(**kwargs)
        except RateLimitError:
            if attempt == retries - 1:
                raise
            time.sleep(min(60, 2 ** attempt + random.random()))

Beyond the retry loop: cap concurrency at the client side so you rarely hit 429 at all; queue and smooth bursty traffic; and treat sustained 429s as a capacity signal (request more quota, add caching, or move part of the workload to a cheaper, higher-limit model) rather than something retries can fix.

Surfacing quota pressure to operators

Because there are no rate-limit headers, visibility must come from your side and from Azure's:

• Count 429s as a first-class metric. Alert on 429 rate over a rolling window, not on individual failures. Rising 429s with flat traffic often means a neighbor workload in the same subscription is consuming the shared pool.
• Track your own token consumption. Every response's usage object tells you the input tokens (cached and uncached) you just spent; aggregating it gives you an ITPM estimate you can compare against your known limit.
• Use the Foundry portal's monitoring. Per-model token and request detail is available in the portal's Monitoring tab — useful for attributing pool consumption across deployments.
• Log request-id and apim-request-id from every response; support requests for throttling anomalies need both.

Design so the limits rarely bite

Teams that handle throttling well treat 429s as the last line of defense, not the first. Separate workloads by model where you can: because pools are per model+version, pointing an internal batch job at a Haiku 4.5 deployment keeps it out of the pool your Sonnet 5 customer traffic depends on — and Haiku's pay-as-you-go defaults are twice as high anyway. Where workloads must share a model, put a gateway or queue in front of the deployment so you can assign priorities yourself; Foundry enforces one shared pool and has no notion of which of your callers matters more. And revisit quota before launches, not during them — increases go through a request form, not a real-time API.

Where to go next

Prompt caching is the cheapest ITPM relief valve — see prompt caching on Foundry. For the broader quota model and increase process, read Foundry quota types and requesting a quota increase.

Sources

• Claude models in Microsoft Foundry (Microsoft Learn)
• Claude in Microsoft Foundry (Anthropic docs)
• Claude billing in Foundry (Microsoft Learn)