Temperature (LLM Sampling)
Temperature scales an LLM's output probabilities to control randomness: low values give focused output, high values give creative variety.
Temperature is a parameter that adjusts how an LLM selects its next token from the probability distribution it computes. It controls the trade-off between predictable, focused output and varied, creative output, without changing the model's weights.
How It Works
Before the model converts its raw scores (logits) into probabilities, those scores are divided by the temperature value. A low temperature, near zero, sharpens the distribution so the most likely token almost always wins, producing deterministic and conservative output. A high temperature flattens the distribution, giving less likely tokens a better chance and increasing diversity and surprise. A temperature of one leaves the distribution unchanged. Temperature is often combined with top-p or top-k sampling to bound the candidate set.
Why It Matters
Temperature is a key control for matching model behavior to a task. Low values suit factual extraction, classification, code, and anything needing consistency or reproducibility. Higher values suit brainstorming, creative writing, and idea generation. Setting it too high can raise the chance of incoherent or hallucinated output, while setting it too low can make responses repetitive.
Related Terms
Temperature is a sampling control used during inference alongside top-p-sampling to shape large-language-model output, and is often tuned together with prompt-engineering.