This playbook describes a reference observability stack for applications calling Claude on any platform. The ingredients — structured logs, a metrics store with histograms, and alerting — are your existing tooling (CloudWatch, Cloud Logging, Azure Monitor, or a vendor). What's specific to LLM apps is which fields to capture, and the Claude API gives you most of them for free in every response.
The structured log schema
Emit one JSON log line per model call, containing at minimum:
| Field | Where it comes from |
|---|---|
trace_id | Your own correlation ID, propagated from the user-facing request through every downstream call |
request_id | The request-id response header (e.g. req_018EeWyX…); error bodies also carry a request_id field. This is what Anthropic support asks for |
model, platform | The exact model ID and which surface served it (1P, Claude Platform on AWS, Bedrock, Vertex, Foundry) — essential once you have fallback routing |
stop_reason | From the response; a drift toward max_tokens or refusal is an early quality signal |
| Token usage | input_tokens, output_tokens, cache_creation_input_tokens, cache_read_input_tokens — total input is the sum of the three input fields |
thinking_tokens | usage.output_tokens_details.thinking_tokens, so reasoning spend is visible separately |
| Latency | Time to first token and total duration (see below) |
| Error fields | HTTP status, error type (e.g. rate_limit_error, overloaded_error), retry count |
Log prompts and completions separately from this operational record, under your data-retention and privacy rules — the operational schema above contains no message content and can be retained freely.
Latency histograms, not averages
Record two timings per streamed call: time to first token (user-perceived responsiveness) and total duration. Streaming responses arrive as server-sent events — message_start, then content-block deltas, then message_delta and message_stop — so first-token time is simply the first content_block_delta. Note that token counts in message_delta usage are cumulative, and that with thinking display set to "omitted" you get faster time-to-first-text but are still billed for full thinking tokens — a reason to plot latency and cost as separate histograms. Use histograms with percentiles (p50/p95/p99); LLM latency is long-tailed and averages hide the tail your users feel.
Two error paths, not one
HTTP-level failures are the familiar path: 429 rate_limit_error, 500 api_error, 529 overloaded_error, and friends. But streaming adds a second path: an error can occur after a successful 200 response begins, arriving as an in-stream error event (for example an overloaded_error mid-stream). If your error dashboards only count HTTP statuses, mid-stream failures are invisible. Instrument both, and tag which retries the SDK performed — official SDKs auto-retry 429s, 5xx, and connection errors with exponential backoff, honoring the retry-after header.
Alert thresholds worth paging on
- Rate-limit saturation: the
anthropic-ratelimit-*response headers report each limit'slimit,remaining, andreset, showing the most restrictive limit in effect. Gaugeremaining/limitand warn at, say, sustained low headroom — before the 429s start. - 429 and 529 rates: a 429 names the exceeded limit and carries
retry-after; sharp traffic spikes can also trip acceleration limits, so alert on ramp rate, not just volume. 529 reflects API-wide load — the documented mitigations are backoff, a less-loaded model, or spreading traffic. - Cache hit-rate drop: a falling ratio of
cache_read_input_tokensto total input usually means someone broke a cached prefix — a silent cost regression worth an alert of its own. - Quality proxies: rising
stop_reason: "max_tokens"or"refusal"share.
anthropic-ratelimit-* headers and usage-tier structure described here are documented for the Claude API; Claude Platform on AWS uses the same rate limits (fixed at the Start tier). Bedrock, Vertex, and Foundry surface throttling through their own quota systems — wire those into the same dashboards via CloudWatch, Cloud Monitoring, or Azure Monitor.Where to go next
This stack is the feedback loop for gateway policies and model rollouts. For platform-specific plumbing, see Bedrock CloudWatch metrics and the observability overview.