Anthropic maintains a migration guide with explicit paths between model generations — Claude Opus 4.8 → Fable 5, Opus 4.7 → 4.8, Opus 4.6 → 4.7, Sonnet 4.6 → Sonnet 5, Mythos Preview → Mythos 5, and more. There is no dedicated "regression testing" chapter; instead, the same verification instructions reappear in every path's checklist. That repetition is the signal. Whether you call Claude through the first-party API, Amazon Bedrock, Vertex AI, or Microsoft Foundry, a model swap is a dependency upgrade, and three steps recur: re-baseline, re-tokenize, re-review.
Step 1: Re-baseline cost and latency on your own workloads
The instruction that appears across every migration path, verbatim: "Re-baseline cost and latency on your own workloads." The emphasis on your own is deliberate. Published per-token prices tell you the rate, not the bill — a new model may think more or less by default, produce longer or shorter answers, and tokenize your prompts differently, all of which move real cost and latency independently of the price card. The practical move: replay a representative sample of production traffic against the new model in staging, and compare tokens per request, dollars per request, and latency percentiles against the same sample on the old model. On third-party platforms, your cloud billing and monitoring surfaces (CloudWatch on AWS, Cloud Monitoring on Google Cloud, Azure Monitor) are where the re-baselined numbers become dashboards.
Step 2: Re-run token counts after a tokenizer change
Token counts are measurements, and measurements are made with a specific tokenizer. The guide is explicit on the Sonnet 5 path: "Re-run token counting against Claude Sonnet 5 rather than reusing counts measured against earlier models." The reason: Opus 4.7 and later Opus models, Fable 5, and Sonnet 5 use a newer tokenizer that produces roughly 30% more tokens for the same text than the previous generation. Any number derived from old counts — cost projections, context-budget arithmetic, truncation thresholds, quota planning — silently drifts unless re-measured. Not every path triggers this step (Opus 4.8 → Fable 5 keeps the same tokenizer, so counts are roughly unchanged), but the checklist item is to check, not to assume. The full story is in tokenizer drift.
Step 3: Review prompts and harness for behavior shifts
The third recurring step is qualitative: a prompt-and-harness review. The guide calls this out especially for the Opus 4.7 migration, pointing at behavior changes in response length, literal instruction following, tone, and tool-usage patterns. A model that follows instructions more literally can break prompts that relied on the old model's looser interpretation — and no error message will tell you. This is where a real eval suite earns its keep: re-run it on the new model and diff the results before users do it for you.
The guide also lists concrete code-path checks alongside the prompt review:
| Check | Why it matters |
|---|---|
Treat the thinking field as display text only; pass thinking blocks back unchanged | Modified thinking blocks are rejected |
Test stop-reason handling, including refusal and model_context_window_exceeded | Newer models emit stop reasons older code may not handle |
| Remove assistant-message prefills | Prefill returns a 400 on current-generation models |
Two model-specific knobs round out the review. Re-evaluate your effort setting — the guide says to start at high even for workloads that ran at xhigh on Opus 4.8 (details in effort retuning). And on prefill, see the affected-model list.
/claude-api migrate applies the model ID swap plus, as needed, breaking parameter changes, prefill replacement, and effort calibration — and then "produces a checklist of items to verify manually." The tool does the mechanical edits; the three steps above are still your job.Where to go next
Platform-specific upgrade wrinkles live in Bedrock surface upgrades and tokenizer breaks on upgrade. For building the eval suite that step 3 depends on, start with automated evals at scale and prompt regression testing.