Unit 1: How LLMs work, for defenders · Lesson 3 of 5 · 16 min
Temperature and why answers change
Watch a model pick its next word by chance, and learn why the same question gets different answers.
mission
- You are
- You are the AI security engineer at Fernway Travel.
- Situation
- A customer posted two screenshots: Fernbot said their refund would be processed, then on a retry that it would be denied.
- Goal
- Explain how that can happen and which setting controls it.
- Hand in
- The setting, its current value and a recommendation.
Temperature as a weighted spinner
Read the sampler
tools/sample.py is a 31 line Python script in your home folder that copies how a model picks its next word. It holds a small table of odds for two prompts. Temperature reshapes the odds: each one is raised to the power 1 / temperature, then they are scaled to add up to 1. random.choices draws a word using those odds, and --seed fixes the random draws so a run can be repeated.
cat tools/sample.py
import argparse
import random
ODDS = {
"your refund will be": {"processed": 0.52, "approved": 0.27, "delayed": 0.13, "denied": 0.08},
"the capital of portugal is": {"Lisbon": 0.97, "Porto": 0.02, "Madrid": 0.01},
}
parser = argparse.ArgumentParser(description="Pick the next word the way a language model samples it.")
parser.add_argument("prompt", choices=ODDS)
parser.add_argument("--temperature", type=float, default=1.0)
parser.add_argument("--seed", type=int, default=7)
parser.add_argument("-n", type=int, default=5)
args = parser.parse_args()
odds = ODDS[args.prompt]
words = list(odds)
if args.temperature == 0:
best = max(words, key=odds.get)
probs = [1.0 if w == best else 0.0 for w in words]
else:
weights = [odds[w] ** (1 / args.temperature) for w in words]
probs = [x / sum(weights) for x in weights]
for word, p in zip(words, probs):
print(f"{word:<10} {p:.2f}")
print("---")
rng = random.Random(args.seed)
for i in range(args.n):
pick = rng.choices(words, weights=probs)[0]
print(f"{i + 1}. {args.prompt} {pick}")
defender tip Python 3 comes with most Linux systems. To try it on your own machine, copy the file and run it with python3.
cat- Prints the contents of one or more files. Built into Linux and macOS.
Temperature 0
At temperature 0 the model always takes the most likely next token. Use it when you want repeatable answers.
python3 tools/sample.py "your refund will be" --temperature 0
processed 1.00
approved 0.00
delayed 0.00
denied 0.00
---
1. your refund will be processed
2. your refund will be processed
3. your refund will be processed
4. your refund will be processed
5. your refund will be processed
python3- Runs Python code or a Python script. Built into most Linux systems. Otherwise from python.org.
Temperature 1
At 1 the model samples from its own odds. Most of the time you get the top word, sometimes not.
python3 tools/sample.py "your refund will be" --temperature 1 --seed 11
processed 0.52
approved 0.27
delayed 0.13
denied 0.08
---
1. your refund will be processed
2. your refund will be approved
3. your refund will be denied
4. your refund will be processed
5. your refund will be processed
you should see Three different endings from the same prompt, and one of them is denied.
Temperature 1.8
Higher temperature flattens the odds. Rare and wrong words become much more likely. The seed is the same, so the random draws are the same. Only the odds changed.
python3 tools/sample.py "your refund will be" --temperature 1.8 --seed 11
processed 0.40
approved 0.28
delayed 0.18
denied 0.14
---
1. your refund will be approved
2. your refund will be approved
3. your refund will be denied
4. your refund will be approved
5. your refund will be approved
you should see denied rises from 8 percent to 14 percent, and approved now wins most draws.
Temperature 0.3
Low values keep the model close to its best guess while allowing small variation.
python3 tools/sample.py "your refund will be" --temperature 0.3
processed 0.89
approved 0.10
delayed 0.01
denied 0.00
---
1. your refund will be processed
2. your refund will be processed
3. your refund will be processed
4. your refund will be processed
5. your refund will be processed
Check the app's setting
Find what the production app uses before you blame the model.
grep temperature app/config.json
"temperature": 0.2,
grep- Prints the lines of text that match a pattern. Built into Linux and macOS.
When the model is sure
When one answer dominates, temperature barely matters. Variation shows up where the model is unsure.
python3 tools/sample.py "the capital of portugal is" -n 3
Lisbon 0.97
Porto 0.02
Madrid 0.01
---
1. the capital of portugal is Lisbon
2. the capital of portugal is Lisbon
3. the capital of portugal is Lisbon
you should see Lisbon every time.
defender tip Never let a sampled sentence decide money or access. Put the decision in code and let the model only explain it.
Who decides the refund
reply = llm("Will this refund be approved?")
send(reply)Different runs can give different decisions
approved = policy.check(booking)
reply = llm(f"Explain: approved={approved}")The model can only phrase a fixed decision
What a defender takes away
- 1LLM output is sampled, so the same input can give different output.
- 2Low temperature reduces variation but does not make the model correct.
- 3Decisions belong in code. The model explains them.
Questions
0 of 3 solved
Answer each one from what you find on your lab machine.