Temperature, Top-P, and When to Touch Them

When Claude generates a response, it repeatedly chooses the next token from a ranked list of candidates. Temperature controls how adventurous that choice is: lower values concentrate probability on the top candidates for steadier, more repetitive output, while higher values spread it out for more varied phrasing. Top-p (also called nucleus sampling) trims the candidate list itself, keeping only the most probable tokens whose combined probability reaches the threshold p. Top-k is a blunter cousin that caps the list at a fixed count. For years, tuning these was standard practice — classifiers at low temperature, brainstorming at high.

The 2026 reality: on new models, the knobs are gone

This is the fact that surprises teams migrating prompts forward: per Anthropic's documentation, the newest models — including Claude Fable 5, Claude Opus 4.8, Opus 4.7, and Claude Sonnet 5 — reject non-default temperature, top_p, and top_k values with a 400 error on every request, whether or not extended thinking is active. A pipeline that has always sent temperature=0 will fail outright when pointed at these models. The fix is to stop sending the parameters, not to find a magic accepted value.

Models outside that list — for example, older generations that support manual thinking budgets — still accept sampling parameters; check the documentation for the specific model you deploy. But the direction of travel is clear, and it applies across platforms, since Bedrock, Vertex AI, and Foundry all expose the same underlying models.

What replaced them

The newer models expose different levers that better match what teams were actually reaching for:

• Wanted determinism or strict formats? Use structured outputs: output_config: {"format": {"type": "json_schema", "schema": {...}}} constrains generation to your schema — a hard guarantee, which temperature=0 never was. For labels, an enum in the schema replaces low-temperature classification.
• Wanted to trade quality against speed and cost? That's now the effort parameter (output_config: {"effort": "..."}), with levels max, xhigh, high (the default — setting it is exactly equivalent to omitting it), medium, and low. Per the docs it shapes all response tokens, and it's a behavioral signal rather than a strict budget.
• Wanted more careful reasoning on hard inputs? Adaptive thinking (thinking: {"type": "adaptive"}) lets the model decide per request whether and how much to deliberate.
• Wanted varied phrasing? Ask for it in the prompt ("propose three headlines with distinctly different angles"). Instructed diversity is more controllable than sampled diversity ever was.

Conservative defaults for enterprise workloads

For teams that want one policy across summarization, extraction, classification, and drafting, the boring answer is the right one:

1. Send no sampling parameters at all. Defaults are the only universally safe configuration across current and older models, and on the newest ones they're mandatory.
2. Put format guarantees in schemas, not in sampling settings — see output format control.
3. Put consistency in the prompt: tight instructions, anchored criteria, and 3–5 examples do more for label and style stability than any temperature ever did.
4. Judge settings by evals, not vibes. Anthropic's prompt-engineering overview is explicit that empirical tests against clear success criteria come first; a sampling tweak you can't measure is a superstition.

If a vendor or an internal runbook insists a workload "requires temperature 0.2," treat that as a claim to test — written, most likely, for a different model generation. On current Claude models the question is settled by the API itself: the parameters are simply not part of the contract anymore.

Where to go next

For the API-level view of sampling and the parameters that remain, see temperature and sampling in the API. The replacements are covered in structured outputs and the broader prompt evaluation framework.

Sources

• Adaptive thinking
• API errors
• Effort
• Structured outputs
• Prompt engineering overview