"Misuse" here spans a range: an external user probing your chatbot with injection attempts, an insider running personal workloads through the company's API keys, a leaked credential burning tokens from an unexpected place, or a legitimate team quietly repurposing an approved app for an unapproved use. Different actors, same detection philosophy — establish what normal looks like, then investigate deviations. Anthropic and the cloud providers run their own platform-level trust-and-safety processes (Anthropic's Usage Policy enforcement can, for example, retain flagged content for extended periods), but those protect the platform, not your policies. Your application layer is yours to watch.
The signals worth watching
Query volume anomalies. Sudden spikes or sustained off-hours traffic on a key or workspace that has had a stable rhythm for months. On the Claude API, the Admin Usage API reports token consumption with breakdowns by API key, workspace, model, and service tier, in time buckets as fine as one minute; data typically appears within about five minutes of request completion, and the documented guidance for sustained polling is once per minute. A spike in 429 rate-limit errors is the same signal arriving from the other direction. On Bedrock, Vertex AI, and Foundry, use your cloud provider's native metering and logging the same way — see audit logging Claude usage.
Output-type shifts. A rising refusal rate is the clearest one: recent Claude models return stop_reason: "refusal" when safety classifiers engage, so the ratio of refusals to normal completions is a one-line metric — a user or integration suddenly generating refusals is either broken or probing. Watch also for drift in output length, language, or topic mix against each lane's baseline.
Unusual user patterns. One account iterating rapidly on near-identical prompts with small mutations (the signature of manual jailbreak search), a service integration suddenly submitting free-form conversational text, sessions that walk through the attack families in your adversarial catalogue, or requests arriving from infrastructure that has never called the app before. These require request-level logs with a caller identity — which your application must record itself, since provider-side inputs and outputs are deleted on Anthropic's backend within 30 days by default.
Cost anomalies. Spend is a slow but reliable proxy. The Admin cost endpoint reports daily costs groupable by workspace; a workspace that doubles its burn without a launch on the calendar deserves a look — overlap here with cost monitoring is a feature, not duplication.
A lightweight detection layer
- Log per-request metadata at your gateway: timestamp, caller identity, key/workspace, model ID, token counts, stop reason, and a policy-relevant classification of the request if you have one. Metadata, not necessarily full content — see prompt log management for the content-retention trade-offs.
- Baseline per lane, per week. Simple rolling averages beat no baselines; you can add statistics later.
- Alert on a handful of rules: volume > N× baseline, refusal rate above a threshold, new caller identity on a production key, spend jump on any workspace. Start noisy, tune down.
- Write the triage step down: who looks, within what time, and what "confirmed misuse" escalates to — your incident playbook and incident response process.
Response levers you already have
Structure access so that containment is cheap. On the Claude API, API keys are scoped to a single workspace; the Admin API can set a key's status to inactive without touching anything else, workspace spend limits cap monthly damage below the organization's limit, and archiving a workspace immediately revokes every key in it (irreversibly — a last resort). One key per integration, one workspace per team or environment, and "disable the key" becomes a two-minute action instead of an outage. Equivalent segmentation on Bedrock, Vertex AI, and Foundry comes from your cloud IAM — separate roles or service accounts per application.
Where to go next
Deepen the statistics side with usage anomaly detection, and connect confirmed findings to content policy enforcement. Pre-launch, red-teaming tells you which behaviors to watch for.