The economics motivate the pattern. At list prices, Claude Haiku 4.5 costs $1 per million input tokens and $5 per million output tokens; Claude Opus 4.8 costs $5 and $25; Claude Sonnet 5 sits between at $3 and $15 ($2/$10 introductory through August 31, 2026); Claude Fable 5, the most capable tier, costs $10 and $50. That is a 5x spread between Haiku and Opus on both sides of the meter. If 70% of your traffic is genuinely Haiku-shaped, routing pays for itself immediately — and these list prices hold across the cloud marketplaces, so the math travels to Bedrock, Vertex AI, and Foundry.
The triage classifier
The standard design puts a small, fast classification call in front: Haiku 4.5 reads the request and labels it — answerable directly, needs the heavyweight model, or needs a human. Structured outputs make the classifier reliable: with output_config: {"format": {"type": "json_schema", ...}} the response is guaranteed to match your schema, so the router parses a label instead of scraping prose. Structured outputs are generally available on the Claude API and supported on Claude Platform on AWS, Vertex AI, Bedrock (a subset), and Microsoft Foundry (beta).
from anthropic import AnthropicVertex
client = AnthropicVertex(project_id="acme-ai", region="global")
schema = {"type": "object", "additionalProperties": False,
"properties": {"route": {"enum": ["direct", "escalate", "human"]}},
"required": ["route"]}
triage = client.messages.create(
model="claude-haiku-4-5@20251001", max_tokens=64,
output_config={"format": {"type": "json_schema", "schema": schema}},
messages=[{"role": "user", "content": f"Classify this request:\n{req}"}],
)
# route == "escalate" -> re-run the task on claude-opus-4-8Two schema-design notes from the docs: set additionalProperties: false, and know that refusals come back as HTTP 200 with stop_reason: "refusal" — treat that as an automatic escalate-to-human signal, since you are billed either way. Also remember model IDs differ by platform: bare IDs on most platforms, anthropic.-prefixed on Bedrock, and older models use dated forms (Haiku 4.5 is claude-haiku-4-5@20251001 on Google Cloud).
Cost–accuracy trade-offs
Choose the tiers empirically: run a labeled sample of production traffic through each candidate model and measure quality against your own rubric before fixing the routing thresholds — never assume where the Haiku/Opus boundary sits for your task. A few documented asymmetries to factor in:
Context. Haiku 4.5 has a 200K-token context window while the larger current models have 1M — long-document requests may need escalation for capacity, not intelligence. Escalation double-spend. An escalated request pays for the Haiku attempt plus the Opus run; if your escalation rate creeps past roughly a third of traffic, re-examine whether Sonnet 5 as the default tier beats Haiku-plus-escalations. Caches don't follow the route. Prompt caches are model-scoped, so a shared system prompt must be cached once per tier; a router that flaps between models on the same conversation rebuilds caches and erases the savings. Route per conversation, not per turn, where you can. Within a single tier you also have a documented knob before switching models: the output_config: {"effort": ...} parameter (levels max to low) trades thoroughness for tokens on supported models — note it is not listed as supported on Haiku 4.5.
Routing under load
Rate limits are enforced per model, which makes a router a load-management tool, not just a cost tool. On the documented Start tier, Opus-class models (a combined bucket across Opus 4.8/4.7/4.6/4.5), Sonnet 5, and Haiku 4.5 each get their own 1,000 RPM / 2M ITPM / 400K OTPM budget — separate pools the router can spill between. Two adaptive behaviors are worth wiring in from day one: on a 429, honor the retry-after header and consider serving the request from another tier rather than queueing; on a 529 overloaded_error (API-wide load), the official guidance is to retry with backoff or move traffic to a less-loaded model. The anthropic-ratelimit-*-remaining response headers give the router a live view of each tier's headroom, so it can shift traffic before hitting the wall rather than after.
Keep the fallback chain honest, though: silently downgrading a request that the classifier marked "needs Opus" trades an outage for a quality incident. A better degraded mode is explicit — queue escalations for retry, or tell the user the full-quality answer is delayed.
Where to go next
See the FinOps view of Haiku routing, model tiering strategy, and prompt caching architecture — the model-scoped cache interaction deserves its own read.