Module 1 · Lesson 3 of 6
Temperature: why answers vary
Temperature reshapes the chances
The model scores each possible next piece, and the temperature in your request decides how strongly those scores pick the winner.
Scene 1 of 5 · Scores become chances
At temperature 1, "order" gets 58 chances in 100 and the other three share the rest.
- Your code — healthy
- Language model — healthy — temperature 1
- Scores become chances. At temperature 1, "order" gets 58 chances in 100 and the other three share the rest.
- Turn it down. At 0.2 the gaps grow: "order" now wins 99 times in 100.
- Down to zero. At 0 the top score wins every draw, though providers still do not promise identical replies.
- Turn it up. At 1.5 the gaps shrink, so a piece other than "order" comes out 53 times in 100.
- Same scores throughout. The scores never changed, so no setting here made the model know more.
- Now you try. Pick a temperature, and see how often the top piece wins. For example: "order" wins 71 times in 100; another piece wins 29 times. A middle setting: mostly the top piece, with some variety in the wording.
Scene 1 of 5: Scores become chances. At temperature 1, "order" gets 58 chances in 100 and the other three share the rest.
Key points
- Every possible next piece gets a score from the model.
- Temperature turns those scores into chances before one piece is drawn.
- Low temperature repeats the top pick; high temperature spreads the picks out.
- Temperature changes how the model picks, not what it knows.
For each next (A piece of text that a language model reads or writes, often part of a word. Prices and limits are counted in tokens.), the model gives every possible piece a score. Then one piece is drawn by chance, weighted by those scores. A high scorer usually wins, but not every time.
(A request setting that controls how much chance goes into picking each next token. Low gives repeatable answers; high gives varied ones.) is a setting in the request that controls how strongly the scores decide. A low temperature lets the top piece win almost every time. A high temperature gives lower scores more turns.
Use a low temperature where there is one right answer, such as pulling an order number from an email. Use a higher one where variety helps, such as ideas for a product name.
A temperature of 0 makes answers far more repeatable, but providers do not promise the same output each time. It does not make the model more correct either: it changes how a piece is picked, not what the model knows.
In system design: Choose the temperature for each task, and do not expect two calls to return the same text. Test answers with checks that allow different wording.