Amazon Bedrock Guardrails is AWS's configurable safety layer for foundation models, including Claude. It offers several policy types you can combine: content filters (covering categories such as Hate, Insults, Sexual, Violence, Misconduct, and Prompt Attack, for both text and images), denied topics, word filters with exact matching and a profanity list, sensitive-information filters that block or mask PII and support custom regex patterns, contextual grounding checks for retrieval-augmented applications, and Automated Reasoning checks that validate responses against logical rules. Guardrails evaluate both the input prompt and the model's completion, and can be attached to an invocation by guardrail ID and version — or run standalone via the ApplyGuardrail API without invoking a model at all.
That flexibility is exactly why go-live testing matters. Every filter you enable is a classifier, and classifiers make two kinds of mistakes: false negatives (harmful content passes) and false positives (legitimate content is blocked). This article describes a practical process for measuring both before your users do.
Build two test sets, not one
Most teams test only the content they want blocked. Test the opposite direction just as hard.
The "should block" set. Collect realistic examples for each policy you enable: off-topic requests for your denied topics, prompts containing the PII types you filter, injection-style prompts if you use the Prompt Attack filter. Write them the way real users would — paraphrased, misspelled, embedded mid-paragraph — not as textbook examples.
The "should pass" set. Sample genuine traffic from your pilot or drafts of typical requests. A customer-service assistant will legitimately discuss refunds, complaints, and angry customers; an HR assistant will legitimately handle names and employment details. These are the prompts most likely to trip an over-tightened filter.
Run evaluations with ApplyGuardrail
The Bedrock console lets you configure a guardrail and try prompts interactively, which is useful for exploratory tuning; check the AWS Guardrails documentation for the current console workflow. For systematic measurement, though, use the standalone ApplyGuardrail API: it evaluates content against a guardrail version without invoking a model, so you can sweep hundreds of test cases quickly and cheaply, record which policy fired on each, and recompute your rates after every configuration change.
Two testing details are easy to miss. First, input tagging — evaluating only selected sections of a prompt rather than the whole thing — is available through the SDK only, not the console, so if your production integration uses tagging, your tests must exercise it through the SDK too. Second, guardrails do not evaluate reasoning content blocks. If you run Claude with thinking enabled, the reasoning content is outside the guardrail's view; test against the final response text, which is what the guardrail actually sees.
Tune iteratively, one policy at a time
When results come back, resist the urge to adjust everything at once. Change one policy, re-run both test sets, and compare. Typical adjustments include reducing a content-filter strength for a category that is flagging your legitimate domain language, narrowing a denied-topic definition that was written too broadly, and switching a sensitive-information filter from block to mask where redaction is acceptable but refusal is not.
Also decide deliberately between the Classic and Standard guardrail tiers. Standard extends detection to harmful content inside code elements — comments, variable and function names, string literals — which matters for developer-facing assistants but may change how code-heavy test cases score. Whichever tier you pick, test on that tier.
Version, then freeze before launch
Guardrails are applied by ID and version. Treat versions the way you treat application releases: create a new version for each tuning round, record the false positive and false negative rates measured against it, and pin production to a specific tested version rather than a moving target. When you later loosen or tighten a policy, re-run the full suite against the new version before switching production over. Keep the test sets in source control alongside the results — six months from now, when someone asks why a filter is set where it is, the numbers are the answer.
Where to go next
For the wider guardrails picture, see the guardrails overview and the deep dives on content filters and PII filtering. The launch checklist covers the rest of your pre-production review.