A moderation pipeline answers one question at high volume: does this content violate our policy? The design goal is not "let AI decide." It is a tiered system where cheap deterministic checks handle the obvious, a fast model classifies the ambiguous middle, and humans review everything the model is unsure about — with an audit trail connecting every decision to its evidence. This article covers defensive screening of content coming into your systems; it is not a guide to generating or evading anything.
Tier the pipeline by cost
Tier 0 — deterministic filters. Blocklists, URL and file-type checks, rate anomalies. Code answers these for free; never spend model tokens on what a regex can decide.
Tier 1 — model classification. Claude Haiku 4.5, at $1 per million input tokens and $5 per million output tokens, is the economical choice for a hot-path classifier: moderation prompts are short and the output is a label. Write the policy into the system prompt as explicit categories with examples — the prompting guidance to wrap 3–5 diverse examples in <example> tags applies directly, and explaining why each rule exists helps the model generalize to novel phrasings. The policy text is identical across every request, so mark it with a prompt-cache breakpoint; cache reads cost one-tenth of the base input price, which matters enormously at moderation volume.
Tier 2 — human review. Anything the model labels as violating, borderline, or belonging to a category you have designated high-stakes goes to a review queue your team works through. The queue is ordinary application infrastructure — a database table with states like pending / upheld / overturned — and its "overturned" rows are your most valuable data: they tell you where the classifier and the policy disagree.
Make the verdict machine-readable
Use structured outputs so the classifier's answer is always parseable JSON rather than prose. Note that response prefilling — the old trick for forcing format — is no longer supported on current models and returns a 400 error; enum-constrained structured output is the modern replacement:
from anthropic import AnthropicFoundry
client = AnthropicFoundry(api_key="...", resource="my-resource")
resp = client.messages.create(
model="claude-haiku-4-5",
max_tokens=300,
system=policy_prompt, # cached policy + examples
output_config={"format": {"type": "json_schema", "schema": {
"type": "object", "additionalProperties": False,
"required": ["verdict", "category", "rationale"],
"properties": {
"verdict": {"enum": ["allow", "review", "block"]},
"category": {"type": "string"},
"rationale": {"type": "string"}}}}},
messages=[{"role": "user", "content": user_content}],
)Two edge cases need handling. A refusal comes back as HTTP 200 with stop_reason: "refusal" and may not match your schema — treat it as a "review" verdict, since content the model declines to process is by definition not cleared. And stop_reason: "max_tokens" means a truncated, possibly invalid response; retry with a higher limit.
Audit trail design
For every decision, persist: the content identifier, a hash of the exact prompt version and policy version used, the model ID, the raw structured verdict, the request ID from the response (every API response carries one, useful for support), and the eventual human outcome if reviewed. Version your policy prompt like code — when the policy changes, you want to know which decisions were made under which policy. This is your evidence base for appeals, for regulators if your industry has them, and for measuring classifier drift. Retention rules for this log are a governance decision; see audit trail design and content policy enforcement.
Batch the backlog, stream the hot path
Real-time screening runs on the standard Messages API. Retrospective jobs — re-screening the archive after a policy change — fit batch processing: Anthropic's Message Batches API takes up to 100,000 requests per batch at 50% of standard prices, with most batches finishing in under an hour. That API is available on the first-party Claude API and Claude Platform on AWS, but not on Bedrock, Vertex AI, or Foundry. On Bedrock and Vertex you are not without options: both offer their own cloud-native batch inference for Claude (S3-based on Bedrock; BigQuery/GCS-based on Vertex at 50% batch pricing). Structured outputs are GA on every surface except Foundry, where they are beta. Bedrock users should also evaluate Bedrock Guardrails as a complementary, AWS-native filtering layer.
Where to go next
See the classification service pattern for the general-purpose version of this design, and escalation to human for review-queue mechanics. The feature matrix shows per-platform support.