Product Catalog Enrichment Pipeline

Catalog enrichment means taking sparse product data — a title, a supplier description, maybe a photo — and producing structured attributes (material, color, size class), a category assignment, and customer-facing copy. It is one of the highest-volume LLM workloads in retail, which makes the architecture less about clever prompting and more about throughput, cost, and catching bad output before it reaches your storefront.

The pipeline shape

A reliable enrichment pipeline has four stages, and only two of them involve Claude:

1. Prepare. Deterministic code assembles one request per SKU: the raw record, your attribute taxonomy, and 3–5 worked examples. Anthropic's prompting guidance recommends relevant, diverse examples wrapped in <example> tags for exactly this kind of repeated extraction task.

2. Extract and classify. Claude returns attributes and a category as guaranteed-shape JSON using structured outputs (below).

3. Validate. Code — not the model — checks taxonomy membership, unit sanity, and banned-claim rules. Anything that fails goes to a retry queue or human review.

4. Load. Validated rows land in your product database with the source request ID kept for traceability.

Guaranteed-shape output

Structured outputs let you attach a JSON Schema to the request via output_config, so responses parse every time instead of "usually." Set additionalProperties: false on objects, and note the documented schema restrictions: no recursive schemas, no numeric minimum/maximum, no string length constraints — enforce those in your validation stage instead. Structured outputs are generally available on the Claude API, Claude Platform on AWS, Amazon Bedrock, and Vertex AI, and in beta on Microsoft Foundry.

from anthropic import AnthropicVertex

client = AnthropicVertex(project_id="acme-prod", region="global")
schema = {"type": "object", "additionalProperties": False,
          "properties": {"category": {"type": "string"},
                         "material": {"type": "string"},
                         "copy": {"type": "string"}},
          "required": ["category", "material", "copy"]}
msg = client.messages.create(
    model="claude-haiku-4-5@20251001", max_tokens=1024,
    output_config={"format": {"type": "json_schema", "schema": schema}},
    messages=[{"role": "user", "content": sku_prompt}])

Haiku 4.5 ($1/$5 per million tokens) handles most attribute extraction well; route ambiguous or high-value SKUs to Sonnet 5 or Opus 4.8. Two response cases need code paths: stop_reason: "refusal" (billed, output may not match the schema) and stop_reason: "max_tokens" (possibly truncated JSON — retry with a higher limit).

Run it as batch, not as a request storm

Catalog enrichment rarely needs real-time answers, and every major surface offers a half-price batch lane — but they are different mechanisms:

With Anthropic's Message Batches, give every request a custom_id (your SKU is the natural choice), poll until processing_status becomes ended, and stream the .jsonl results file. Most batches finish in under an hour; requests unfinished at 24 hours expire and are not billed, and results stay downloadable for 29 days. Put shared instructions and your taxonomy behind a prompt-cache breakpoint with the 1-hour TTL, since batches can outlive the default 5-minute cache.

Quality controls that scale

Schema validity is not accuracy. Three controls keep quality visible at 500,000 SKUs: sample-based human review (a fixed random percentage per run, tracked over time), a cheap second-pass grading call on a sample asking a model to score attribute correctness against the source text, and hard business-rule checks in code (price-adjacent claims, restricted terms, category-attribute compatibility). Keep humans in the loop for new categories until the sampled error rate stabilizes — see human-in-the-loop design.

Where to go next

Read the Message Batches deep dive for lifecycle details, structured outputs basics for schema design, and batch vs realtime for the economics.

Sources

• Structured outputs
• Batch processing
• Prompt caching
• Prompting best practices