Google Vertex AI in Practice

Prompt Caching on Vertex AI: Setup and Token Savings

If your requests share a long prefix — a system prompt, a policy document, a tool catalogue — prompt caching cuts the cost of every repeat to a tenth. On Vertex AI you enable it explicitly, per content block.

Claude 3P 101 · Updated July 2026 · Unofficial guide

Prompt caching lets Claude store the processed form of a prompt prefix and reuse it across requests instead of reprocessing the same tokens every time. For enterprise workloads — where thousands of requests a day share the same system prompt, policy text, or few-shot examples — it is typically the single biggest cost lever available. Both cache durations (5 minutes and 1 hour) are generally available for Claude on Google Vertex AI, including on the global endpoint.

One Vertex-specific caveat first

On Anthropic's first-party API, a convenience called automatic prompt caching can manage cache placement for you. That automatic mode is not supported on Vertex AI. On Vertex you use the classic mechanism: explicit cache_control markers on individual content blocks in the request body. (Despite the folk name "caching headers," nothing goes in HTTP headers — the marker is part of the JSON payload.) Cache diagnostics, a first-party debugging feature, is also unavailable on Vertex, so your visibility comes from the usage fields on each response.

Setting a cache breakpoint

Place your stable content first — system prompt, reference documents, tool definitions — and mark the end of the stable prefix with cache_control. Everything up to the marker becomes cacheable; the per-user content after it stays dynamic:

from anthropic import AnthropicVertex

client = AnthropicVertex(project_id="my-project", region="global")
resp = client.messages.create(
    model="claude-opus-4-8", max_tokens=1024,
    system=[{"type": "text",
             "text": LONG_POLICY_DOCUMENT,          # stable prefix
             "cache_control": {"type": "ephemeral"}}],
    messages=[{"role": "user", "content": user_question}])
print(resp.usage)  # cache_creation_input_tokens / cache_read_input_tokens

The first request pays a cache write; subsequent requests with an identical prefix, arriving within the cache lifetime, pay the much cheaper cache read. Two gotchas: the prefix must match exactly (a one-character edit is a miss), and changing the model string invalidates the cache — the first request on a new model writes it fresh.

Minimum cacheable size

Prompts below a per-model minimum are processed normally even if marked. On Claude Opus 4.8 the documented minimum is 1,024 tokens. Claude Fable 5 has a lower 512-token minimum on the first-party API (Bedrock keeps 1,024 for it); Anthropic's docs don't state a Vertex-specific figure for Fable 5, so budget on 1,024 and verify against current documentation. In practice this threshold rarely matters: prefixes worth caching are usually thousands of tokens.

The economics, in numbers

Google's Vertex pricing for Claude applies the standard multipliers to each model's input price:

OperationMultiplierOpus 4.8 example (per 1M tokens)
Regular input1x$5.00
5-minute cache write1.25x$6.25
1-hour cache write2x$10.00
Cache hit (read)0.1x$0.50

The break-even math is friendly: the 5-minute cache pays for itself after a single hit, and the 1-hour cache after two. Beyond break-even, every hit replaces a 1x charge with a 0.1x charge — a 90% saving on the cached portion.

Measuring what you're saving

Each response's usage object splits input into regular input_tokens, cache_creation_input_tokens, and cache_read_input_tokens. Log all three and track your cache hit rate — read tokens as a share of read-plus-regular. A low hit rate usually means the "stable" prefix isn't actually stable (timestamps, user names, or reordered tools sneaking in) or traffic is too sparse for the 5-minute lifetime, in which case the 1-hour duration or request batching by tenant helps. One accounting note: cached tokens still occupy the context window — caching changes what you pay, not what fits.

Rule of thumb: structure every prompt as stable prefix → cache marker → dynamic suffix, and treat cache hit rate as a first-class production metric.

Where to go next

For the cross-platform economics, read Prompt Caching: The Single Biggest Cost Lever; for turning hit rate into an optimization program, see the cache hit rate strategy guide.

Sources