Cost Optimization & FinOps

Tool Call Token Overhead: The Hidden Cost of Agent Loops

An agent that makes ten tool calls doesn't pay for one prompt — it pays for ten, and each one re-reads everything that came before. Here is where the tokens actually go.

Claude 3P 101 · Updated July 2026 · Unofficial guide

When you give Claude tools — the ability to call functions your code executes — you pay for tokens you never wrote. Anthropic's pricing documentation is unusually specific about this, and the numbers matter because agent loops multiply them. A single chat request pays the overhead once. A twenty-step agent run pays it twenty times, on top of a conversation history that grows with every tool result.

Three layers of overhead

1. The tool-use system prompt. The moment your request includes any tools, the API silently prepends instructions that teach the model how to call them. The size depends on the model and on your tool_choice setting (whether the model may decide to use a tool, or is forced to). Official figures, in input tokens:

Modelauto / noneany / tool
Claude Opus 4.8290410
Claude Sonnet 5354474
Claude Haiku 4.5496588

If no tools are provided at all, tool_choice: none adds zero tokens.

2. Tool definitions. Every tool's name, description, and JSON schema is sent as input tokens on every request in the loop. Anthropic's own built-in tools give a sense of scale: the bash tool definition adds 325 input tokens on Opus 4.7/4.8 (244 on earlier models), the Claude 4.x text editor tool adds 700, and the computer-use tool adds 735 plus a 466–499-token system prompt addition. Custom tools with verbose descriptions and deep schemas can easily exceed those figures.

3. Result injection. When your code returns a tool result, it becomes part of the conversation and is billed as input tokens on the next turn — and on every turn after that. A tool that returns a 5,000-token database dump on step 2 of a 15-step run gets re-read (and re-billed) roughly 13 more times. Official docs are explicit that everything in the request counts toward the context window: system prompt, all messages including tool results, tool definitions, and generated output including thinking.

Rule of thumb: in a long agent loop, the cost driver is rarely the model's answers. It's the ever-growing input prefix — schemas plus accumulated tool results — replayed on every step.

Why the growth is quadratic-ish

Each turn's input includes all previous turns. If every step adds a similar amount of content, total input tokens across an N-step run grow roughly with N², not N. A run that looks affordable at 5 steps can be surprising at 25. Adaptive thinking adds a further wrinkle: thinking tokens are billed as output, and on recent models previous thinking blocks are kept in context by default, counted as input on later turns.

What actually reduces the overhead

Cache the stable prefix. Prompt caching (available on all four 3P platforms) bills cache reads at 0.1x the base input price. Tool definitions and the system prompt sit at the top of the cache hierarchy (toolssystemmessages), so they are ideal cache candidates — but note that changing tool definitions invalidates all cached levels. On Bedrock you must place explicit cache_control breakpoints; automatic caching is not supported there.

Trim schemas. Shorter descriptions, fewer optional parameters, and compact enums cut a per-request tax. See optimizing tool schemas for concrete techniques.

Trim results before returning them. Your code controls the tool result payload. Return the rows the model needs, not the full response body. Server-side context editing (clear_tool_uses_20250919) can also clear old tool results automatically once input exceeds a trigger threshold — with the caveat that clearing invalidates cached prefixes at the clearing point.

Count before you ship. The token counting endpoint accepts tools and system prompts and returns an estimate for free, so you can measure the fixed overhead of your exact tool set per model. Counts are model-specific: Opus 4.7+, Fable 5, and Sonnet 5 use a newer tokenizer that produces roughly 30% more tokens for the same text than earlier models.

A quick worked magnitude check

Suppose an agent on Claude Sonnet 5 (standard list price $3 per million input tokens) carries 3,000 tokens of tool schemas and system prompt, and runs 12 steps in which accumulated history averages 20,000 input tokens per step. That's roughly 240,000 input tokens ≈ $0.72 per run before any output — versus about $0.024 for those same 240,000 tokens if most of the prefix is served from cache at $0.30 per million. The arithmetic is why cache hit rate, not model choice, is often the first lever for agent workloads.

Where to go next

Read the cache break-even math to price the caching strategy, and cost regression testing to catch schema growth in CI. The feature matrix shows which caching modes each platform supports.

Sources