When Claude uses client tools, your application executes every call. That puts you in a perfect position to memoize: if a tool is deterministic — same input, same output, at least for a while — there is no reason to hit the external system again. This pattern layers application-side memoization (your code, your cache) with Anthropic's prompt caching (which cuts the token cost of results already sitting in the conversation). They solve different problems and work best together.
Layer 1: Memoize in your tool executor
This layer is standard engineering practice, not an API feature. In the loop that handles tool_use blocks, key a cache on (tool_name, canonicalized_input) and return the stored tool_result on a hit. Recommended practice:
Classify tools by determinism. A currency-table lookup is cacheable for hours; "get current queue depth" is not. Set a TTL per tool, in code.
Canonicalize inputs before hashing. Sort JSON keys and normalize whitespace, or trivially different argument orderings will miss. Adding strict: true to tool definitions helps here too, since inputs are guaranteed to match your schema exactly.
Never cache errors. A tool_result you returned with "is_error": true should be retried, not replayed.
Share across sessions deliberately. A Redis-style shared cache lets every session benefit, but only for tools whose results carry no per-user data — scope user-specific results to the session key.
Because client-tool execution is always your code, this layer works identically on all four 3P platforms: Claude Platform on AWS, Amazon Bedrock, Google Vertex AI, and Microsoft Foundry.
Layer 2: Prompt-cache the results already in context
Memoization saves the external call; it does not save tokens, because past tool results are resent as input on every loop iteration. Prompt caching fixes that: tool results in message content are explicitly cacheable, and cache reads bill at 0.1x the base input price (5-minute-TTL writes at 1.25x, 1-hour writes at 2x). Place a cache_control breakpoint so the accumulating conversation — tool results included — is served from cache.
Two documented mechanics matter for agent loops. First, caching is a strict prefix match: one changed byte invalidates everything after it, so keep tool results byte-stable (your memoization layer helps — a replayed result is identical by construction, whereas re-fetching might change a timestamp and silently break the cache). Second, each breakpoint looks back at most 20 content blocks, and a long agentic turn can add more than that; the docs' guidance is an intermediate breakpoint roughly every 15 blocks. You get up to 4 explicit breakpoints per request.
Platform availability: prompt caching works on the Claude API, Claude Platform on AWS, Google Cloud, and Microsoft Foundry; Bedrock supports explicit breakpoints only, with no automatic caching.
Layer 3: Keep bulk results out of context entirely
For tools that return large payloads the agent only needs to filter or aggregate, programmatic tool calling goes further: Claude writes code in the server-side code-execution container that calls your tools, and intermediate results return to the running code rather than the model's context — only the final output reaches Claude. Enable it per tool with "allowed_callers": ["code_execution_20260120"]. Availability is narrower: GA on the Claude API and Claude Platform on AWS, beta on Foundry, and not available on Bedrock or Vertex AI.
And when the cache should be emptied
Old tool results eventually stop paying rent. Context editing (clear_tool_uses_20250919, beta on all five surfaces) clears stale tool results server-side once input tokens pass a trigger, keeping the most recent N. Note the interaction: clearing invalidates cached prefixes at the clearing point, so use clear_at_least to make each invalidation worth it, and exclude_tools for results that must survive.
Where to go next
See prompt cache mechanics, programmatic tool calling, and caching for high-traffic apps.