You cannot reliably count Claude tokens by rule of thumb anymore. Recent models — Sonnet 5, Opus 4.7 and later, Fable 5 — use a newer tokenizer that produces roughly 30% more tokens for the same text than the previous generation, so any "four characters per token" heuristic calibrated last year now under-counts. The dependable approach is to ask the platform: Claude on Vertex AI supports token counting as a first-class capability (Google documents a Claude token-counting page under its partner-model section), and the anthropic SDK exposes it against the same client you already use for messages.
The pre-flight call
from anthropic import AnthropicVertex
client = AnthropicVertex(project_id="my-project", region="global")
count = client.messages.count_tokens(
model="claude-sonnet-5",
system="You are a contract-review assistant.",
messages=[{"role": "user", "content": long_document}],
)
print(count.input_tokens)Pass exactly what the real request will contain — system prompt, full message history, tool definitions — because all of it counts toward the context window. Two limits to remember: counting tells you input tokens only (output isn't knowable in advance — budget it via max_tokens), and the count is per-model, since tokenizers differ across generations. If your SDK version's method shape differs, check the current documentation rather than guessing.
Use 1: estimate cost before you spend it
With an input count in hand, cost estimation is arithmetic against Google's pricing page (global endpoint, per 1M tokens): Sonnet 5 at $3 input / $15 output ($2/$10 promotional through August 31, 2026), Opus 4.8 at $5/$25, Haiku 4.5 at $1/$5, Fable 5 at $10/$50. A 120,000-token contract bundle into Opus 4.8 is $0.60 of input before the model writes a word. One Vertex-specific threshold matters enormously here: on Google's pricing, a request whose input context is 200K tokens or longer is charged at long-context rates for all tokens, input and output. A pre-flight count is the only clean way to know which side of that line a request lands on — and 199K vs 201K tokens is a very different line item.
Use 2: avoid oversized payloads and context overflows
Vertex rejects request payloads over 30 MB, and context windows are hard limits: 1M tokens for Fable 5, Opus 4.8/4.7/4.6, Sonnet 5, and Sonnet 4.6 on this platform; 200K for Haiku 4.5 and other older models. Sending an over-limit request costs you a round trip and an error in production. Counting first lets the application degrade gracefully instead — truncate, summarize earlier turns, or route the request to a 1M-context model rather than failing. This is the difference between "the model politely handled a huge document" and a 400 at 2 a.m.
Use 3: gate requests at runtime
In a multi-tenant service, token counting becomes a policy point. A gate like the one below enforces per-request ceilings before quota, budget, or latency are affected:
MAX_INPUT = 150_000 # policy ceiling, below long-context threshold
def guarded_call(messages):
n = client.messages.count_tokens(
model="claude-sonnet-5", messages=messages).input_tokens
if n > MAX_INPUT:
raise ValueError(f"Request is {n} tokens; limit {MAX_INPUT}")
return client.messages.create(model="claude-sonnet-5",
max_tokens=2048, messages=messages)Gating interacts directly with quota: Vertex meters Claude in requests and tokens per minute, shared per model lineage per location for recent models, so one user's 400K-token request draws down the same bucket as everyone else's traffic. Google's quota documentation also carries a useful caveat — token usage shown on the console Quota page may be inaccurate due to Anthropic's token estimation and refund system, and it points you to the token counting API or token_count metrics in Metrics Explorer for accurate numbers. In other words, the pre-flight count isn't just a guardrail; it's the accurate measurement.
Where to go next
For what a token actually is, start at Tokens 101. For the billing side, see Vertex billing mechanics and budget alerts for Claude spend; for the throttling side, quota types.