If you've read the defensive introduction to red-teaming and decided to run an exercise, the scoping document is the deliverable that makes it worthwhile. It fits on two pages and has four parts: goals, roles, rules of engagement, and a definition of "reportable finding." This article walks through each, assuming you are testing your own Claude-based application with your organization's authorization — defensive testing, nothing else.
Goals: testable claims, not vibes
"See if the chatbot can be jailbroken" is not a goal; it has no finish line. Write goals as falsifiable claims about your controls, each traceable to a policy or design decision: "the assistant never reveals system prompt content," "no tool call executes against a record the authenticated user cannot access," "generated replies never state discount terms beyond published policy." Five to eight claims is a healthy first scope. Anything not listed is out of scope by default — which keeps the team off tangents and keeps the report answerable: each claim ends the exercise as held, broken (with evidence), or untested.
Roles: who does what
| Role | Responsibility |
|---|---|
| Sponsor | Authorizes the exercise in writing, owns the risk decision on findings. Usually the application owner's leadership or the governance body. |
| Lead | Writes the scope, runs the schedule, arbitrates "is this in scope?", assembles the report. |
| Testers (2–4) | Execute attack families from your adversarial prompt catalogue. Include at least one person who didn't build the system. |
| Application owner | On call to explain intended behavior and confirm whether an observation is actually a violation. |
| Security/compliance observer | Watches for scope drift and handles anything that must escalate to incident response. |
Rules of engagement
Rules of engagement (RoE) define the sandbox. The essentials:
- Target and environment: the exact deployment under test — a staging instance, never production, and never with real customer data as test input.
- Credentials and isolation: testers use exercise-issued credentials only. On the Claude API side, a dedicated workspace works well: its API keys are scoped to that workspace, you can set a workspace spend limit below the organization's as a hard budget cap, and usage is separately reportable by workspace through the Admin Usage API. Archiving the workspace when the exercise ends immediately and irreversibly revokes all of its API keys — a clean shutdown.
- Time window: fixed start and end dates, so logs are interpretable and monitoring teams know what they're seeing.
- Techniques: adversarial prompting of your own application is in scope; attacking the underlying platform, other tenants, or provider infrastructure is not, and provider acceptable-use terms continue to apply throughout.
- Stop conditions: if testers reach data belonging to real customers or another system, they stop, preserve evidence, and escalate — that is an incident, not a finding to keep pulling on.
- Logging: every attempt recorded verbatim with timestamp, model ID, and prompt version; the pinned model snapshot in the log is what makes findings reproducible later.
What counts as a reportable finding
Define this before testing starts, or the report will be a pile of screenshots. A reportable finding has four properties: it is reproducible (a recorded prompt sequence that reliably triggers the behavior, on a named model ID and prompt version); it violates a stated claim or policy, not merely a tester's taste; it has a severity rating agreed against a simple scale (e.g., policy embarrassment / data exposure / unauthorized action); and it names the control that failed — input handling, prompt design, tool authorization, output filtering. Interesting-but-in-spec behaviors go in an observations appendix, not the findings list.
After the exercise
The lead delivers findings with severities to the sponsor; the sponsor decides fix-before-launch versus accept-with-monitoring — explicitly, in writing. Every broken claim's reproduction prompts join the permanent regression suite, documented per your eval record practice, and get re-run after fixes and after every future model or prompt change.
Where to go next
Stock the testers' toolkit with the adversarial prompt catalogue, and wire persistent detection with misuse monitoring.