By default, an API call to Claude returns nothing until the entire response is generated. For a long answer that can mean tens of seconds of silence — long enough for users to assume the app is broken and for intermediate proxies to give up on the connection. Streaming fixes both problems by delivering the response incrementally, token by token, as it is generated. Streaming for Claude is generally available on Google Vertex AI.
What happens on the wire
Vertex AI exposes two endpoint verbs for Claude. Non-streaming requests go to :rawPredict; streaming requests go to the same URL with :streamRawPredict, and the response comes back as server-sent events (SSE) — a simple HTTP convention where the server keeps the connection open and emits a sequence of small event messages. Each event carries a piece of the response: content deltas as text is generated, plus lifecycle events marking the start and end of the message and of each content block. You rarely construct these URLs yourself; the SDKs select the right verb automatically.
Streaming with the Python SDK
The AnthropicVertex client — installed with pip install -U google-cloud-aiplatform "anthropic[vertex]" — provides a streaming helper that manages the SSE connection and yields clean text chunks:
from anthropic import AnthropicVertex
client = AnthropicVertex(project_id="my-project", region="global")
with client.messages.stream(
model="claude-sonnet-5",
max_tokens=2048,
messages=[{"role": "user", "content": "Summarize our Q2 results memo..."}],
) as stream:
for text in stream.text_stream:
print(text, end="", flush=True)
final = stream.get_final_message() # complete Message, incl. usageThe text_stream iterator is the ergonomic path for chat-style UIs. If you need the raw event objects — for example, to detect tool-use blocks mid-stream or to forward events to a browser — iterate over the stream object itself and switch on each event's type. The final accumulated message gives you token usage for cost tracking, exactly as a non-streaming call would.
Node and other languages
For TypeScript and Node, Anthropic publishes a dedicated package: npm install @anthropic-ai/vertex-sdk. It authenticates through the standard google-auth-library flow — the same Application Default Credentials story as Python — and exposes equivalent streaming helpers, so the event-handling pattern carries over. If you are calling the REST endpoint directly from another language, you consume the :streamRawPredict SSE stream with any standard SSE client.
Timeouts: three clocks to align
Streaming changes which timeouts matter. Three separate clocks are ticking on a streamed request:
| Clock | Failure mode if too short |
|---|---|
| SDK / HTTP client timeout | The client aborts a healthy stream mid-response on long generations |
| Intermediaries (load balancers, gateways, proxies) | An idle or long-lived connection is cut before the model finishes |
| Your application's own request budget | Upstream callers time out waiting for your service to finish relaying |
The Anthropic SDKs accept a configurable request timeout; set it generously for streaming endpoints, because with SSE the meaningful health signal is "are events still arriving?", not total elapsed time. Then audit every hop between the user and Vertex — API gateways and load balancers frequently default to 30–60 second idle limits that silently kill long streams. Specific timeout defaults vary by SDK version, so check the official SDK documentation for current values.
Operational notes
Streaming changes error handling in one important way: once the first bytes have gone out, there is no clean way to retry transparently — the user has already seen half an answer. Handle failures before the stream starts (auth, quota, validation) with normal retries, and treat a mid-stream disconnect as its own case: either surface a "response interrupted" state or restart the generation explicitly. For accounting, log the usage figures from the final message event exactly as you would for a non-streaming call — streamed tokens are billed identically, and quota consumption is the same either way. Finally, if your service relays the stream onward to browsers, propagate backpressure rather than buffering the whole response in memory; the point of streaming is lost if a middle tier collects everything before forwarding it.
max_tokens is more than a few hundred, and always read usage from the final message event rather than estimating.Where to go next
For why streaming is the default UX posture, read Streaming Responses: Better UX and Fewer Timeouts. Streaming interacts with tool use — covered in Tool Use with Claude on Vertex AI — and with thinking output, covered in Extended Thinking on Vertex AI.