CloudWatch's generative AI observability feature bundles pre-built views for two things: Model Invocations (inference traffic against Bedrock models — the view relevant to Claude API calls) and Bedrock AgentCore agents (latency, usage, errors, and end-to-end prompt tracing for agentic workloads). For an enterprise team that wants a "how is Claude doing today" screen without designing dashboards, this is the fastest path on AWS.
What the Model Invocations dashboard shows
AWS documents that the Model Invocations dashboard works with any models used for inference in Amazon Bedrock. Out of the box it displays:
- Invocation count and latency,
- Token counts by model, and daily token counts by ModelID,
- Requests grouped by input-token ranges — a quick read on your request-size distribution,
- Throttles and error counts.
That covers the operational basics: volume, cost drivers (tokens), capacity pressure (throttles), and health (errors). It is built on the same metrics you could chart yourself from the AWS/Bedrock namespace — the dashboard's value is assembly, not new data. Which also means it inherits that namespace's blind spot: traffic on the current bedrock-mantle endpoint publishes to a separate AWS/BedrockMantle namespace, so check coverage for your surface mix (see the namespace guide).
The prompt-content drill-down needs invocation logging first
The dashboard includes an invocation table with per-request drill-down into what was actually sent and received. This is the part teams expect to "just work" and it doesn't: viewing invocation input and output content requires that you first enable Bedrock model invocation logging with CloudWatch Logs as a destination. The dashboard reads the content from those log records and links per-request drill-downs into CloudWatch Logs Insights.
Two consequences follow:
- The bedrock-runtime-only caveat applies. Invocation logging captures only
Converse/InvokeModel-family calls on thebedrock-runtimeendpoint. Calls made through the current Claude in Amazon Bedrock surface are not captured, so their content can never appear in the invocation table regardless of configuration. - You are now storing prompts. Once logging is on, full request and response bodies live in CloudWatch Logs. AWS itself suggests applying CloudWatch Logs data-protection (masking) policies for sensitive content in invocation logs. Decide who can read that log group before you flip the switch, not after — prompts routinely contain customer data. Prompt log privacy covers this in depth.
Extending with OpenTelemetry traces
The metrics-and-logs view answers "what happened per request." For agentic applications — multi-step chains where one user action fans out into several Claude calls and tool invocations — you want traces. CloudWatch gen-AI observability ingests structured traces from third-party frameworks (AWS documents Strands, LangChain, and LangGraph) instrumented with the AWS Distro for OpenTelemetry (ADOT) SDK, or via the AWS Lambda Layer for OpenTelemetry. Bedrock AgentCore goes further: its telemetry has built-in support for OTel and the GenAI semantic conventions — the emerging standard attribute names for model calls (provider, model, token usage) that make traces portable across observability vendors.
The practical pattern: keep the pre-built dashboard as your fleet-level view, and add ADOT instrumentation to the specific applications where per-conversation tracing matters. The two meet in CloudWatch, so an on-call engineer can go from "error spike on this model" to "this specific chain failed at step three" without switching tools.
Where to go next
For alarming rather than viewing, start from the Bedrock CloudWatch metrics reference and the log-delivery health metrics. For a cross-platform view of observability stacks, see the observability stack pattern.