When an LLM generates a response, it doesn't just pick the single most likely next word — it samples from a probability distribution over possible words. This randomness is controlled by a setting called temperature. A higher temperature makes responses more varied and creative; a lower temperature makes them more predictable and consistent.

This sampling approach is why the same prompt can yield meaningfully different outputs. The model has many plausible ways to continue a sentence, and each run through the distribution is its own random draw.

Interestingly, sampling more from a single model at different temperatures is not the same as getting responses from truly different models. Research shows there's a dimensionality gap — varying temperature explores a narrow slice of possible outputs, while genuinely diverse models trained differently cover far more ground. Randomness and diversity are not the same thing.