Audit Trail Design for LLM Pipelines

An audit trail for an LLM pipeline answers three questions after the fact: what went in, what came out, and who or what acted on it. None of the four platforms builds this for you end to end — the API gives you the identifiers and usage data; the immutable store, redaction, and retention policy are your architecture. The design below is recommended practice, hedged where compliance is involved: your deployment inherits your cloud provider's compliance posture, and retention rules for your industry should be confirmed with your provider and counsel, not inferred from a guide.

What to capture per request

Log one record per model call, written by the same gateway or wrapper that makes the call, containing:

Correlation identifiers. Every Claude API response carries a request-id header (and error bodies include a request_id field) — this is the ID Anthropic support can act on, so it belongs in every record. Add your own trace ID, the caller's identity, feature tag, and timestamp.

The exact request. Model ID, full rendered prompt (system, messages, tool definitions), and parameters. "Version 12 of the prompt template" is not enough when templates interpolate data — store what was actually sent, or a hash plus a retrievable copy.

The exact response. Content blocks, stop_reason, and the full usage object (input, cache creation, cache read, output tokens) for cost reconstruction.

Tool activity. For agentic pipelines, every tool_use block and every tool_result you returned. Anthropic's own security guidance for the bash tool says to log every command the model runs — extend that principle to all tools, since tool calls are where an LLM system touches the outside world.

Making the log immutable

An audit trail you can edit is a liability with extra steps. Recommended practice: write records append-only to object storage with write-once/read-many protections and restrictive delete permissions under your cloud's IAM, hash-chain or timestamp batches if your auditors expect tamper evidence, and separate the writing identity from every reading identity. All three clouds offer native primitives for this; the pattern is the same on each, so stay within whichever your platform already uses. If you use Managed Agents (beta, first-party API and Claude Platform on AWS only — not Bedrock, Vertex, or Foundry), some of this arrives built in: every memory-store mutation produces an immutable version with an actor record, and sessions persist full event history server-side until you delete them.

Redaction: log pointers, not payloads

The tension is real — a faithful audit trail wants everything; privacy wants minimal personal data with deletion rights. A two-tier design resolves it: an audit tier holding metadata, identifiers, hashes, usage, and tool call summaries (long retention, no raw personal data), and a payload tier holding full prompts and responses encrypted, keyed by record ID (shorter retention, deletable). Deleting a payload honors an erasure request while the audit tier still proves the interaction happened — the same principle the Managed Agents memory system applies, where leaked secrets can be redacted from a version while the audit history is preserved. Run deterministic PII detection before write where feasible, and never log credentials: Anthropic's guidance warns that secrets placed in prompts persist in session history, which is equally true of your own logs.

Retention: know the platform's clocks, set your own

Your own retention schedule — how long each tier lives, who approves disposal — is a policy decision. Implement it as scheduled lifecycle rules in code, and log the disposals themselves.

Where to go next

Pair this with platform audit logging, PII handling, and the PII redaction pipeline.

Sources

• API errors (request IDs)
• Bash tool (logging guidance)
• Batch processing (results retention)
• Files API (deletion behavior)
• Managed Agents overview