When a language model generates text, it repeatedly chooses the next token from a probability distribution. Temperature scales how adventurous that choice is: lower values concentrate probability on the most likely tokens (more consistent, more repetitive), higher values flatten the distribution (more varied, less predictable). Top-p — "nucleus sampling" — takes a different cut: it restricts choices to the smallest set of tokens whose combined probability reaches the threshold p. Top-k simply limits choices to the k most likely tokens. Historically, the practical advice was to adjust temperature or top_p for a given workload, not both at once, since they push on the same behavior from different angles.
The 2026 reality: the newest models reject these parameters
Here is the fact that surprises teams migrating old code: on Claude Fable 5, Claude Opus 4.8, Opus 4.7, Sonnet 5, and the Mythos models, sending temperature, top_p, or top_k at non-default values returns a 400 invalid_request_error — on every request, regardless of whether extended thinking is active. The migration guides are blunt about the fix: remove the parameters entirely. Older currently-supported models, including Claude Haiku 4.5, Sonnet 4.6, and Opus 4.6, still accept them.
| Model | temperature / top_p / top_k |
|---|---|
| Fable 5, Opus 4.8, Opus 4.7, Sonnet 5 | Rejected with 400 at non-default values — omit them |
| Haiku 4.5, Sonnet 4.6, Opus 4.6 and earlier | Accepted |
This applies wherever the model runs — the parameters are a property of the model generation, not of the platform, so the same 400 comes back on Bedrock, Vertex AI, and Foundry. If you operate a mixed fleet (say, Haiku 4.5 for cheap classification and Sonnet 5 for generation), your request-building code must be model-aware: include sampling parameters only for models that accept them.
Guidance where the knobs still exist
On models that accept sampling parameters, the traditional mapping to business tasks still holds. For classification and extraction — anything where two runs on the same input should agree — push temperature down toward its minimum. For generation tasks where variety is a feature (drafting subject lines, brainstorming alternatives), a higher temperature earns its keep. One honest caveat belongs in every runbook: even at the lowest temperature, large language models are not guaranteed to be bit-for-bit deterministic across runs, so consistency-critical pipelines should validate outputs rather than assume identical ones. For exact parameter ranges and defaults, check the official API reference for your model.
What replaces sampling control on the newest models
Anthropic's newest models pair the removal of sampling knobs with stronger, more direct controls — and for business workloads these are usually better tools than temperature ever was:
Structured outputs. For extraction and classification, output_config.format with a JSON schema guarantees the response parses and conforms — a much stronger consistency guarantee than any temperature setting. Strict tool use ("strict": true on a tool definition) does the same for tool calls, and forcing a specific tool via tool_choice turns classification into schema-guaranteed selection.
Effort. The output_config.effort parameter (low through max) controls how much work the model invests — thinking depth, tool-call thoroughness, response length. It is not a randomness dial, but it is the supported way to tune the cost/latency/quality trade-off that teams often reached for temperature to manage.
Prompting. Instructions about format, length, and variability remain fully effective — telling the model to answer with a single label, or conversely to propose three distinct options, shapes output more reliably than sampling tweaks.
A migration checklist
Before switching a workload to a Fable 5-generation model: grep your codebase and prompt-orchestration config for temperature, top_p, and top_k; remove them (or gate them per model); replace determinism-motivated temperature: 0 usage with structured outputs where a schema fits; and re-run your evals — output character can shift between model generations independent of sampling settings.
Where to go next
See the prompting view of temperature and top-p, strict tool use, and thinking budgets and effort levels for the modern control surface.