What is LLM Temperature?

The knob that makes the model focused or creative.

Updated 2026-04-18
Short answer

Temperature is a sampling parameter that controls how random an LLM's token selection is. At temperature 0 the model picks the single most probable next token every time (greedy decoding, fully deterministic); at higher values it samples from the probability distribution, yielding more varied output. Typical ranges are 0-0.3 for factual tasks, 0.5-0.8 for general chat, and 0.8-1.2 for creative writing. Temperature does not change what the model knows — only how it chooses words.

In depth

Every LLM produces a probability distribution over the next token at each step. 'The capital of France is' might put 99% on ' Paris', 0.5% on ' Lyon', and tiny probabilities on thousands of others. Temperature rescales these probabilities before sampling. At temperature 0 the model picks the argmax (always ' Paris'). At temperature 1 it samples from the distribution as-is. At temperature 2 it flattens the distribution, making low-probability tokens more likely. Mathematically, temperature divides the logits before softmax: higher temperature = flatter distribution = more variety. For most production applications the useful range is 0 to 1.5. Below 0.2 the output is nearly deterministic — useful for classification, structured data extraction, and anything where you want the same question to always produce the same answer. 0.3 to 0.7 is the default for assistants and chatbots — enough variety that repeated questions don't produce identical responses, but still tight enough to be reliable. 0.8 to 1.2 is creative writing territory, where you want fresh phrasing and unexpected ideas. Above 1.5 output becomes incoherent. Temperature interacts with two related sampling parameters. Top-p (nucleus sampling) truncates the distribution to the smallest set of tokens whose probabilities sum to p — typically 0.9-0.95. Top-k keeps only the k highest-probability tokens. Most production APIs expose both. The conventional advice is to vary temperature OR top-p, not both simultaneously, because they interact in confusing ways. OpenAI docs explicitly recommend this. For most use cases, temperature alone is the right knob. Temperature has important edge cases. (1) At temperature 0, the output is deterministic in theory but not always in practice — floating-point nondeterminism, batch-size effects, and MoE routing can produce different outputs for the same prompt at temperature 0 across different inference runs. True reproducibility requires the vendor to guarantee it (OpenAI's seed parameter, for example). (2) Reasoning models (Claude 4.5 thinking, OpenAI o1/GPT-5 thinking, DeepSeek R1) often ignore temperature because their training schedule fixes it — the reasoning tokens are generated with a specific temperature that the user can't override. (3) Chain-of-thought with low temperature can lock into a bad reasoning path; self-consistency sampling benefits from temperature 0.6-0.8. Common anti-patterns: setting temperature to 0 and assuming the model will be factually correct (it won't — it's just repeating its most probable generation, which can be confidently wrong), setting temperature high because 'creative' seems desirable for code (it isn't — code benefits from low temperature to avoid syntax errors), and forgetting to set temperature in production so you inherit whatever the provider default is (usually 1.0, often too high for factual tasks). For AI agents, temperature is a per-task decision. Tycoon's agents use different temperatures for different calls. Tycoon Agent uses low temperature (0.2-0.3) for structured status reports and high temperature (0.8) for brainstorming strategic options. The AI developer uses 0.1 for code generation to minimize syntax errors and higher temperature for explaining code to the user. Getting this right is small but compounding — the wrong temperature silently makes an agent either boring or unreliable.

Examples

  • Temperature 0: asking 'what is 2+2' and always getting '4' — deterministic, correct, boring
  • Temperature 0.3: customer-support replies — reliable answers with a touch of natural variation
  • Temperature 0.7: ChatGPT default — good balance of coherence and liveliness for chat
  • Temperature 1.0: creative writing prompts — unexpected word choices, vivid phrasing, occasional weirdness
  • Temperature 1.5: intentionally weird generation for brainstorming or artistic exploration
  • Self-consistency CoT: sample 10 responses at temperature 0.7, take majority answer — works because of temperature-driven variety
  • Code generation at temperature 0.1: minimizes syntactic hallucination while keeping deterministic enough to debug
  • Tycoon Tycoon Agent uses temperature 0.2 for structured reports, 0.7 for strategic exploration, 0.3 for customer-facing copy
what is llm tokenwhat is hallucinationwhat is chain of thoughtwhat is llm streamingwhat is fine tuningwhat is agent evaluation
AI CTO
FAQ

Frequently asked questions

Clear answers about wallet credit, usage, subscriptions, and how Tycoon charges for work.

Does temperature 0 give me reproducible results?

In theory yes, in practice usually not. At temperature 0 the model always picks the highest-probability token, so the output should be deterministic. But real deployments have floating-point nondeterminism on GPU, batch-size effects (what else is in the batch affects scheduling), and for mixture-of-experts models, router nondeterminism. OpenAI and Anthropic both publish caveats about this. For true reproducibility use the provider's seed parameter (OpenAI supports this). For 'close enough' reproducibility — same answer 95% of the time — temperature 0 is fine. For production tests where you assert exact output equality, don't rely on temperature alone.

Should I always use temperature 0 for factual tasks?

Mostly yes, but not always. Low temperature (0-0.2) is right when you want consistent, focused answers — extraction, classification, structured output. Going all the way to 0 sometimes hurts because the top token occasionally happens to be wrong (e.g., a subtle formatting difference) and the model locks in. Temperature 0.1-0.3 gives you nearly-deterministic behavior with tiny variation that can catch edge cases. For QA and RAG, 0.1 is a common sweet spot. For code generation, 0 or 0.1. For creative factual tasks like 'write a business plan,' 0.3-0.5 lets the model vary phrasing while still staying grounded.

What's the difference between temperature and top-p?

Temperature rescales the entire probability distribution before sampling. Top-p (nucleus sampling) truncates the distribution to the top tokens whose probabilities sum to p, then samples from those. High temperature + low top-p: the model is restricted to the top probable tokens but samples flatly among them. Low temperature + high top-p: the model has access to the full distribution but strongly favors the top token. In practice they produce similar-feeling output and you should adjust one at a time. The OpenAI docs recommend not modifying both simultaneously. Default advice: start with temperature alone, reach for top-p only if you've already hit its limits.

Why is higher temperature considered 'more creative'?

Because creativity in language is partly about taking less-obvious word choices. At low temperature the model picks the most probable token at each step, which produces highly-templated, cliché-heavy output. At higher temperature the model picks less-probable but still-plausible tokens, producing more varied phrasing, unexpected turns, and novel combinations. The tradeoff is coherence — beyond temperature 1.2 or so, outputs start to derail mid-sentence. For genuine creative writing, a moderate temperature (0.8-1.0) with good prompting beats extreme values. 'Creative' doesn't mean maximally random.

Does temperature affect hallucination?

Yes, but less than people assume. Higher temperature increases variance in the output, including variance in the errors — a hallucination at temperature 0 and a hallucination at temperature 1 are differently wrong but both wrong. Low temperature does not prevent hallucination; it just makes it more consistent. The root cause of hallucination is that the model's highest-probability next token is confidently wrong when the question falls outside its reliable knowledge. Temperature modulates how often you see the most-probable answer vs alternatives, not whether that answer is true. For reducing hallucination you need RAG, grounding, and refusal training — temperature is a minor factor.

Run your company with humans and AI agents.

Hire your AI team in 30 seconds. Start for free.

Sign up / Sign inLearn the category

Free to start · No credit card required · Set up in 30 seconds