Solution Patterns & Playbooks

Accuracy-First Extraction: Confidence Scores and Fallbacks

When extracted data feeds invoices, claims, or contracts, a wrong value costs more than a missing one. This pattern makes the model say how sure it is — and routes the unsure cases to people.

Claude 3P 101 · Updated July 2026 · Unofficial guide

Basic extraction asks Claude to turn a document into JSON and hopes for the best. Accuracy-first extraction adds three things: a schema the output must satisfy, a per-field confidence signal, and a defined path for what happens when confidence is low. The goal is a pipeline where every value in your database is either machine-verified or human-verified — never merely plausible.

Pass 1: schema-constrained extraction

Structured outputs (output_config with a JSON schema) guarantee the response parses and contains the fields you declared. Two schema-design decisions carry most of the accuracy weight. First, make every field nullable in spirit: give the model an explicit way to say "not present" rather than forcing an invention. Second, ask for field-level confidence — and because the schema feature does not support numeric range constraints (minimum/maximum return a 400 error), express confidence as an enum, which is also what language models are better calibrated for:

from anthropic import AnthropicBedrockMantle

field = {"type": "object", "additionalProperties": False,
    "required": ["value", "confidence", "source_quote"],
    "properties": {"value": {"type": ["string", "null"]},
        "confidence": {"enum": ["exact_match", "inferred", "uncertain", "absent"]},
        "source_quote": {"type": ["string", "null"]}}}
schema = {"type": "object", "additionalProperties": False,
    "required": ["invoice_number", "total_amount", "due_date"],
    "properties": {k: field for k in ["invoice_number", "total_amount", "due_date"]}}
client = AnthropicBedrockMantle(aws_region="us-east-1")
msg = client.messages.create(
    model="anthropic.claude-sonnet-5", max_tokens=2048,
    output_config={"format": {"type": "json_schema", "schema": schema}},
    messages=[{"role": "user", "content": doc_text}])

(If you prefer Pydantic models over raw schemas, the Python SDK's client.messages.parse() helper accepts them, auto-transforms unsupported schema constraints, and validates the response client-side.) The source_quote field is a cheap verification hook: a value the model cannot quote from the document deserves suspicion.

Pass 2: independent verification

Multi-pass verification means checking extracted values with a separate request rather than trusting one shot. Two documented mechanics make the second pass stronger than the first. Deterministic checks come first, in code: does the quoted text actually appear in the document? Do line items sum to the total? Is the date parseable? Then, for fields that pass code checks but carry low confidence, run a verification call with citations enabled on the document block — citations return the exact character or page location backing each claim, and cited text does not count toward output tokens. One constraint shapes the architecture here: citations cannot be combined with structured outputs in the same request (the combination returns a 400 error). That is precisely why this is a two-pass pattern — pass 1 gets you the schema-valid JSON, pass 2 gets you the grounded evidence.

Rule of thumb: code verifies format and arithmetic; the model verifies grounding; humans verify whatever is left. Never let the same model call both produce and approve a value.

Escalation for low confidence

Define the routing table up front, per field, not per document: exact_match plus passing code checks flows straight through; inferred triggers the verification pass; uncertain or a failed verification lands in a human review queue with the document, the extracted value, and the quote highlighted. Also handle two API-level outcomes explicitly: a refusal returns HTTP 200 with stop_reason: "refusal" (output may not match your schema — route the document to review, don't retry blindly), and stop_reason: "max_tokens" means truncated output — retry with a larger limit. Track the human-override rate per field over time; a rising rate on one field usually means the documents changed, not the model.

Cost and platform notes

Two-pass extraction doubles calls but not necessarily cost: the document is the shared prefix of both passes, so a prompt-cache breakpoint lets the verification pass read it at one-tenth the input price. High-volume backfills fit batch processing at 50% of standard prices — via Anthropic's Message Batches API on the first-party API and Claude Platform on AWS, or via each cloud's own batch inference on Bedrock (S3-based) and Vertex AI (BigQuery/GCS-based, 50% batch pricing). Structured outputs and strict tool use are GA on the Claude API, Claude Platform on AWS, Bedrock, and Vertex AI, and beta on Foundry; citations are GA on all five surfaces. PDFs work on all five surfaces, base64-only on Bedrock and Google Cloud.

Where to go next

See extraction to database for the persistence half, output validation and self-correction for retry mechanics, and form processing for the document-heavy variant. The feature matrix covers platform support.

Sources