Most production Claude workloads are a mix: interactive traffic that must answer in seconds, and background work — nightly report generation, document backfills, evaluation runs, re-embedding pipelines — that merely has to finish by morning. When both draw from the same tokens-per-minute pool, the background work is effectively stealing capacity from users. Moving it to batch gives it back, and cuts its cost in half at the same time.
Why batch relieves the real-time pool
On the Claude API, the Message Batches API has its own rate limits, entirely separate from the synchronous Messages limits: at the Start tier, 1,000 batch-API requests per minute and 200,000 batch requests in the processing queue (rising to 4,000 / 500,000 at the Scale tier — published figures, subject to change). Tokens processed inside a batch do not draw down your real-time ITPM/OTPM. Shift a nightly 50-million-token job from synchronous calls to a batch, and those 50 million tokens vanish from your real-time consumption entirely — headroom your interactive traffic can immediately claim.
The trade is latency: batches are asynchronous. Most complete in under an hour; the hard boundary is 24 hours, after which unfinished requests expire — and expired requests are not billed. A single batch can hold up to 100,000 requests or 256 MB, whichever comes first, and results stay downloadable for 29 days. All usage in a batch is charged at 50% of standard prices: Opus 4.8 drops to $2.50/$12.50 per million tokens, Haiku 4.5 to $0.50/$2.50 (list prices, subject to change).
The platform map — one API, three mechanisms, one gap
Here the platforms genuinely diverge, and wording matters.
| Platform | Batch option | Notes |
|---|---|---|
| Claude API | Anthropic Message Batches API | 50% discount; separate rate-limit pool; stacks with prompt caching |
| Claude Platform on AWS | Anthropic Message Batches API | Supported (IAM action CreateBatchInference); scoped per workspace |
| Amazon Bedrock | AWS's own batch inference | S3-based, asynchronous, 50% lower than on-demand; no Anthropic Message Batches API |
| Google Vertex AI | Google's own batch prediction | Input via BigQuery table or JSONL in Cloud Storage; 50% of on-demand pricing; no Anthropic Message Batches API |
| Microsoft Foundry | None | Message Batches API not available at all |
Note the distinction: Bedrock and Vertex do not offer Anthropic's Message Batches endpoint, but each provides its own cloud-native batch mechanism at 50% of on-demand pricing — it is wrong to say those platforms "have no batch option." The mechanics differ from Anthropic's, though. Bedrock batch inference runs through the bedrock-runtime endpoint (its quotas are separate from both real-time surfaces) and does not support tool calling or structured output. Vertex batch prediction defaults to 4 concurrent batch requests per project and does not work on the global endpoint. Foundry has no batch path — non-urgent Foundry workloads either eat real-time quota or run on another platform.
Practical batching for capacity relief
A few operational details from Anthropic's documentation keep the valve working smoothly. Each request in a batch needs a unique custom_id to match results, which arrive as a .jsonl stream in any order. Because batches can take longer than five minutes, use the 1-hour prompt-cache duration for shared context rather than the 5-minute default — batch and caching discounts stack. And when a burst of urgent work arrives mid-batch, you can cancel: already-processed requests return partial results, and only succeeded requests are billed.
Capacity planners should also note one quirk: on the Claude API, batches may slightly exceed a workspace's configured spend limit because of concurrent processing — relevant if you use workspace limits as hard cost fences.
Where to go next
See the Batch API deep dive for request anatomy, batch vs real-time for the decision framework, and long-context TPM trade-offs for the workload most worth offloading.