What is LLM Temperature?

Every LLM produces a probability distribution over the next token at each step. 'The capital of France is' might put 99% on ' Paris', 0.5% on ' Lyon', and tiny probabilities on thousands of others. Temperature rescales these probabilities before sampling. At temperature 0 the model picks the argmax (always ' Paris'). At temperature 1 it samples from the distribution as-is. At temperature 2 it flattens the distribution, making low-probability tokens more likely. Mathematically, temperature divides the logits before softmax: higher temperature = flatter distribution = more variety. For most production applications the useful range is 0 to 1.5. Below 0.2 the output is nearly deterministic — useful for classification, structured data extraction, and anything where you want the same question to always produce the same answer. 0.3 to 0.7 is the default for assistants and chatbots — enough variety that repeated questions don't produce identical responses, but still tight enough to be reliable. 0.8 to 1.2 is creative writing territory, where you want fresh phrasing and unexpected ideas. Above 1.5 output becomes incoherent. Temperature interacts with two related sampling parameters. Top-p (nucleus sampling) truncates the distribution to the smallest set of tokens whose probabilities sum to p — typically 0.9-0.95. Top-k keeps only the k highest-probability tokens. Most production APIs expose both. The conventional advice is to vary temperature OR top-p, not both simultaneously, because they interact in confusing ways. OpenAI docs explicitly recommend this. For most use cases, temperature alone is the right knob. Temperature has important edge cases. (1) At temperature 0, the output is deterministic in theory but not always in practice — floating-point nondeterminism, batch-size effects, and MoE routing can produce different outputs for the same prompt at temperature 0 across different inference runs. True reproducibility requires the vendor to guarantee it (OpenAI's seed parameter, for example). (2) Reasoning models (Claude 4.5 thinking, OpenAI o1/GPT-5 thinking, DeepSeek R1) often ignore temperature because their training schedule fixes it — the reasoning tokens are generated with a specific temperature that the user can't override. (3) Chain-of-thought with low temperature can lock into a bad reasoning path; self-consistency sampling benefits from temperature 0.6-0.8. Common anti-patterns: setting temperature to 0 and assuming the model will be factually correct (it won't — it's just repeating its most probable generation, which can be confidently wrong), setting temperature high because 'creative' seems desirable for code (it isn't — code benefits from low temperature to avoid syntax errors), and forgetting to set temperature in production so you inherit whatever the provider default is (usually 1.0, often too high for factual tasks). For AI agents, temperature is a per-task decision. Tycoon's agents use different temperatures for different calls. Astra uses low temperature (0.2-0.3) for structured status reports and high temperature (0.8) for brainstorming strategic options. The AI developer uses 0.1 for code generation to minimize syntax errors and higher temperature for explaining code to the user. Getting this right is small but compounding — the wrong temperature silently makes an agent either boring or unreliable.