Batch Analytics Over Archives with Claude

Archive analytics is the friendliest LLM workload there is: every document is independent, nobody is waiting on a spinner, and failures are cheap to retry. That profile maps directly onto batch processing, where all usage is charged at 50% of standard API prices.

Pick your batch surface by platform

Anthropic's Message Batches API is available on the first-party Claude API and Claude Platform on AWS, but not on Amazon Bedrock, Google Vertex AI, or Microsoft Foundry (where it isn't available at all). That doesn't mean no batch options on Bedrock or Vertex — AWS and Google Cloud each offer their own cloud-native batch inference for Claude, S3-based on Bedrock and BigQuery/GCS-based on Vertex AI (the latter at 50% batch pricing). The pattern below is written against the Message Batches API; the checkpointing and cost logic transfers to the cloud-native mechanisms with different plumbing.

Message Batches mechanics that shape the design

A batch holds up to 100,000 requests or 256 MB, whichever comes first — so a million-document archive is a series of batches, not one.
Each request needs a unique custom_id (1–64 characters). Use your document ID; it's the join key for everything that follows.
Most batches finish in under an hour; anything unfinished at 24 hours expires, and expired requests are not billed.
Results stream from the batch's results_url as JSONL, downloadable for 29 days — persist them promptly; the results, not the batch object, are your data.
Individual results are succeeded, errored, canceled, or expired; only succeeded requests are billed.

Checkpointing: a status table, nothing fancier

Keep one row per document: doc_id, batch_id, status, result_location. The driver loop is then restartable from any crash:

from anthropic import AnthropicAWS  # Claude Platform on AWS

client = AnthropicAWS()  # AWS_REGION + ANTHROPIC_AWS_WORKSPACE_ID env vars
pending = db.docs_where(status="pending")[:100_000]
batch = client.messages.batches.create(requests=[
    {"custom_id": d.id,
     "params": {"model": "claude-haiku-4-5", "max_tokens": 1024,
                "messages": [{"role": "user", "content": prompt(d)}]}}
    for d in pending])
db.mark(pending, status="submitted", batch_id=batch.id)

A second loop polls for processing_status: "ended", downloads the JSONL, and updates each row from its custom_id. Requeue errored and expired rows; after two or three failed attempts, park a document in an exceptions state for inspection instead of retrying forever.

Parallelism is queue depth, not threads

The API does the fan-out — you control throughput by how many requests you keep in flight. Batch rate limits are shared across models: at the Start tier, 1,000 batch-API requests per minute and 200,000 requests in the processing queue (300,000 at Build, 500,000 at Scale). A simple governor — "top the queue back up to N whenever a batch ends" — is all the orchestration most archives need.

Cost controls, in order of leverage

Pilot on a slice. Run a few hundred representative documents, check quality and measure real token usage, then multiply. The free token-counting endpoint lets you estimate before spending anything.
Right-size the model. Batched Haiku 4.5 is $0.50/$2.50 per million input/output tokens versus $2.50/$12.50 for batched Opus 4.8. A common shape: Haiku pass over everything, Opus pass over the flagged subset.
Cache the shared prefix. If every request repeats the same instructions or reference material, mark it with cache_control — and because batches can take longer than 5 minutes, the docs specifically recommend the 1-hour cache duration for shared context in batches. Cache reads are billed at 0.1x base input.
Mind PDFs. Pages cost both text tokens (typically 1,500–3,000 per page) and image tokens; the PDF docs explicitly recommend the Batch API for high-volume PDF processing.
Know the spend-limit caveat. Batches may slightly exceed a workspace's configured spend limit due to concurrent processing — don't treat the limit as a hard cap for budgeting.

Where to go next

For the decision between batch and real-time surfaces, see batch vs. real-time; for the Bedrock and Vertex alternatives, the Bedrock workaround and the Vertex workaround. Structured extraction schemas for the per-document prompt are covered in extraction to database.

Sources

← All articles