Programming Fundamentals · Search and Decision

Greedy and Sampling: Two Personalities When AI Picks Words

In the LLM Fundamentals Part you saw the candidate probability bars in an AI’s head as it continues token by token, and you twiddled Temperature. That was “feel”; this lesson names the algorithms: always pick the biggest = greedy; roll dice by probability = sampling. Same starter, each personality generates three times—you’ll see the difference at a glance.

Same starter · two personalities

The starter is fixed: “The weekend plan is”. Play the top machine first: greedy mode (temperature=0), always the longest probability bar. Hit “Regenerate” three times, watch the stacked results below—even one different character?

😤 Greedy mode

temperature = 0

The weekend plan is

Candidates each step—it always takes the longest bar

No dice, no suspense: highest probability wins

Now the bottom machine: sampling mode—each step rolls a weighted die. At 0.7 click thrice and see if results differ; then try 1.2—watch the bar shapes: higher temperature flattens them, so rare words get a better shot.

🎲 Sampling mode

Roll by probability

temperature

The weekend plan is

Probabilities after temperature—dice roll on these

Same candidate table, dice pick the path—each run may fork differently

Concept card · the ledger of two personalities

😤 Greedy: take the biggest right now

Fast, stable, reproducible—ten thousand runs, same sentence. Fatal blind spot: local best each step may not sum to global best. Pick the smoothest first word and the whole line may hit a dead end. The fix? Next lesson’s Beam Search is the antidote—cliffhanger for now.

🎲 Sampling: leave a door for randomness

Trade certainty for diversity. Roll by probability: high-prob words win often (still fluent); low-prob words occasionally surface (surprises). Temperature is the “surprise” budget: higher T flattens the distribution so rare words dare to show.

Interview one-liner: temperature = 0 is greedy decoding—fully deterministic; higher T flattens softmax so the model dares low-prob words—diversity and “going off the rails” rise together. That knob in the LLM Fundamentals Part sits between greedy and sampling.

You’ve met greedy people in life: at a buffet every round they grab the priciest plate—salmon, steak, king crab—until the stomach’s full and the signature wagyu carving never happens. Step-best ≠ whole-best; stomachs are finite, and so is the context window.

What this lesson wants to share

Greedy = pick the highest probability each step: three runs, identical—fast and reproducible, but may miss a better whole
Sampling = roll by probability: three runs, three flavors—trade certainty for diversity
Temperature is the knob between them: T=0 is greedy; higher T flattens the distribution and dares rarer words
Choose by task: reproduce experiments, write code, extract data → T=0; brainstorm or copy → raise T
Greedy’s pit gets filled next lesson: Beam Search—look a few steps ahead before choosing