Here is the honest framing for this audit: max_tokens doesn't change what a well-behaved request bills — you pay for tokens generated, not the ceiling. What right-sizing fixes is the two failure directions around "well-behaved." Over-allocated endpoints let misbehaving requests generate thousands of unneeded tokens at full output price before anything stops them. Under-allocated endpoints truncate answers mid-thought, which shows up as quality bugs and paid-for retries. The audit finds both, endpoint by endpoint.
Step 1: Get the ground truth
You need, per endpoint (per distinct task calling Claude, whatever your architecture calls it): the configured max_tokens, the distribution of actual usage.output_tokens, and the rate of stop_reason: "max_tokens". Sources, from best to workable:
Your own logs. Every response carries usage and stop_reason. If you log them tagged by endpoint, the audit is a query. If you don't — start today; this is the single highest-value observability line for LLM cost work.
Platform reporting. On the first-party Claude API, the Usage Admin API breaks token consumption down by API key, workspace, and model — assign one API key or workspace per service and the breakdown maps to endpoints. On the cloud platforms, use the provider's tooling: CloudWatch metrics on Bedrock, or cost views like Cost Explorer. Note the Usage and Cost API is not available on Claude Platform on AWS; lean on response-level logging there.
from anthropic import AnthropicVertex
client = AnthropicVertex(project_id="my-project", region="global")
msg = client.messages.create(
model="claude-opus-4-8",
max_tokens=1500,
messages=[{"role": "user", "content": prompt},],
)
log.info("endpoint=%s out=%d stop=%s", name,
msg.usage.output_tokens, msg.stop_reason)Step 2: Read the distribution, not the average
For each endpoint, compare the high percentiles of actual output against the ceiling.
| Audit finding | Diagnosis | Action |
|---|---|---|
| p99 output far below the ceiling; no max_tokens stops | Over-allocated — ceiling provides no real circuit-breaking | Lower toward p99 plus honest headroom |
Frequent stop_reason: "max_tokens" | Under-allocated or over-generating | Raise the ceiling, or trim the prompt/effort so answers fit |
| Long right tail on a task that should be short | Prompt invites rambling | Fix the prompt (labels only, structured outputs) — then lower the ceiling |
| Output dominated by thinking tokens | Reasoning spend, not answer length | Tune effort / thinking budget, not just the ceiling |
The last row matters on adaptive-thinking models: thinking tokens count toward max_tokens, and usage.output_tokens_details.thinking_tokens separates reasoning from answer. An endpoint whose "long outputs" are actually deliberation needs the thinking dial, and clamping the ceiling instead will truncate answers.
Step 3: Quantify the correction
Where the tail is real generation you don't want, the savings are concrete. Example with listed prices: a triage endpoint on Claude Sonnet 5 (introductory output price $10/MTok through August 31, 2026) runs 1M requests a month with max_tokens=8000. Logs show the intended answer is ~60 tokens, but 2% of requests ramble to ~4,000 tokens. Those stragglers generate 2% × 1M × 4,000 = 80M tokens — $800/month at $10/MTok — versus 2% × 1M × 600 = 12M tokens, or $120/month, if they were capped near 600 tokens. Fixing the prompt and lowering the ceiling to 600 converts a fat tail into a bounded, observable failure mode. (After September 1, 2026, at Sonnet 5's standard $15/MTok output price, the same tail costs $1,200/month.)
stop_reason: "max_tokens", and re-run the audit whenever you change model or prompt — Opus 4.7+, Fable 5, and Sonnet 5 use a tokenizer that yields roughly 30% more tokens for the same text, so old baselines drift.Step 4: Make it stick
Kill the shared constant. Put max_tokens (and effort) in per-endpoint configuration with an owner, and treat a ceiling change like any config change — reviewed, deployed, observable. Two structural notes as you tighten: batched requests require max_tokens ≥ 1, and on the rate-limit side there's no penalty for headroom — output-per-minute limits count generated tokens only, never the ceiling.
Where to go next
For the reasoning behind the ceilings, read output length budgeting. To cut the other side of the bill, run the same audit mindset on context: summarize vs. full history.