The purpose of a high-stakes classification is operational: it tells an engineer, at design time, whether a given output lane must pass through a mandatory human gate before anything happens (see designing human oversight). To do that job, the definition has to be written down, testable against concrete examples, and owned by someone — usually the same body that runs your use-case intake and risk tiering.
Four dimensions that define "high-stakes"
| Dimension | Question to ask | Pushes toward review when… |
|---|---|---|
| Effect on individuals | Does the output influence a decision about a specific person? | Credit, employment, housing, insurance, benefits, health, legal standing, discipline. |
| Audience | Who sees or relies on it? | External parties, regulators, the press, or any audience that will treat it as the company's word. |
| Reversibility | If wrong, can it be undone cheaply? | Money moves, records change, messages send, commitments bind. |
| Domain sensitivity | Is the subject matter regulated or safety-relevant? | Medical, legal, financial advice; safety instructions; anything your industry regulator watches. |
A practical decision rule: if any single dimension scores high, the output is high-stakes and requires pre-action human review. Don't average across dimensions — a perfectly reversible output that affects someone's employment is still high-stakes. Note that some jurisdictions define high-risk AI uses in law, and those definitions may be broader or narrower than yours; align the internal rubric with your counsel rather than treating this article's framing as a regulatory conclusion.
Categories that should default to mandatory review
Most organizations converge on a similar core list: outputs that recommend or execute adverse decisions about individuals; external communications published under the company's name; anything customers could reasonably rely on as professional advice; content touching regulated claims (financial performance, health benefits, legal rights); and outputs that trigger irreversible system actions. Two less obvious additions earn their place quickly: model refusals — recent Claude models can return stop_reason: "refusal", and a refusal in a production lane deserves human attention rather than a silent retry — and novel input types the system wasn't evaluated on, which are high-stakes by ignorance until your eval record says otherwise.
Making the definition consistent in practice
A rubric nobody can apply consistently produces the worst of both worlds: friction and false confidence. Three habits help:
- Write examples, not just criteria. For each product team, list five concrete outputs that are high-stakes and five that are not. Disagreements about the examples surface ambiguity while it is still cheap.
- Default up. When a team is unsure which side of the line an output falls on, it is high-stakes until the owning body rules otherwise. Ambiguity resolved by the engineer under deadline pressure always resolves downward.
- Classify lanes, not individual outputs. The classification attaches to an output type at design time ("all outbound claim-denial letters"), so the routing is code, not per-item judgment. Runtime signals can escalate items upward, never downward.
Review requirements worth writing down
For each high-stakes lane, specify: who is qualified to review (role, not name); what the reviewer must see (input and output together); the maximum time an item may wait; what happens on rejection; and where the disposition is recorded — which should be your decision audit trail. Revisit the classification whenever the audience, the model, or the prompt changes materially; a lane that was low-stakes as an internal draft tool becomes high-stakes the day its output starts going to customers.
Where to go next
For the lanes that don't require mandatory review, the automate-vs-review framework covers graduated options, and human-in-the-loop patterns shows implementations.