Rebaselining Tool-Schema Token Overhead When Upgrading Model Versions

When you give Claude tools, two hidden token costs ride along on every request: a system-prompt addition that teaches the model how to call tools, and the token cost of the tool definitions themselves. Neither appears in your prompt text, but both are billed as input tokens and both count toward the context window. Crucially, they are model-version-specific — an upgrade can shrink or grow them substantially, which silently shifts per-request cost and how much room is left for your actual content.

The overhead changes more than you'd expect

Anthropic publishes the tool-use system prompt overhead per model. The variation across one upgrade path is striking: with tool_choice of auto or none, Claude Opus 4.7 adds 675 tokens, while Claude Opus 4.8 adds just 290 — an upgrade that reduces fixed overhead by more than half. Sonnet 5 adds 354. The any/tool choices cost more across the board.

Anthropic-defined tools have their own per-model definition costs on top: the bash tool adds 325 input tokens on Opus 4.7/4.8 but 244 on Opus 4.6, Sonnet 4.6, and earlier; the text_editor_20250429 tool adds 700; computer use adds 466–499 system-prompt tokens (beta) plus a 735-token tool definition on Claude 4.x models. And with no tools provided at all, tool_choice: none adds zero.

The tokenizer moves underneath you too

There's a second, larger effect stacked on top. Claude Opus 4.7 and later Opus models, Sonnet 5, and Fable 5 use a newer tokenizer that produces roughly 30% more tokens for the same text (the exact increase depends on content). Your own tool schemas — names, descriptions, JSON Schema parameter definitions — are text, so a 5,000-token tool block on Sonnet 4.6 does not stay 5,000 tokens on Sonnet 5. Published overhead numbers and your own schema sizes must both be re-measured against the target model, not extrapolated.

Re-measure with the token counting endpoint

You don't have to estimate any of this. The token counting endpoint is free, accepts exactly the same inputs as message creation — including tools and system prompts — and returns model-specific counts. Diff the same payload across the two model IDs:

from anthropic import Anthropic

client = Anthropic()
payload = dict(system=SYSTEM_PROMPT, tools=TOOLS,
               messages=[{"role": "user", "content": "ping"}])

for model in ("claude-opus-4-7", "claude-opus-4-8"):
    n = client.messages.count_tokens(model=model, **payload)
    print(model, n.input_tokens)

Counts are estimates — actual billed input can differ slightly, and some system-added tokens appear in counts but are never billed — but they're accurate enough to catch a 30% swing. The endpoint has its own request-per-minute limits (2,000 RPM on the Start tier), separate from message creation, so a one-off audit script won't eat your production quota. It works on Bedrock and Vertex AI too (beta on Foundry), so you can validate the platform you actually run on; just use the right ID form per platform (anthropic.claude-opus-4-8 on Bedrock, bare IDs elsewhere).

Why this bites in production

Two failure modes show up in real systems. First, cost underestimates: a finance model that assumed Sonnet 4.6 token counts will under-forecast a Sonnet 5 workload by roughly the tokenizer delta plus the overhead delta, multiplied by every request. Second, context-limit surprises: agentic systems that pack conversations close to the window on the old model can start overflowing after upgrade, because the same text plus the same tools now occupies more of the window. Both are avoidable with an afternoon of measurement.

Where to go next

See the token counting endpoint for mechanics, the tokenizer break on upgrade for the ~30% shift in depth, and cache pre-warming for the other cost event that lands on upgrade day.

Sources

• Claude pricing (tool use system prompt overhead)
• Token counting
• Model migration guide (tokenizer change)