Chapter Summary
Summary (Part B) · Mitigation Strategies + Decision Framework
Chapter 1 · Review · Part B: how to mitigate hallucinations, and the decision framework every AI PM should carry.
3 · Four Mitigation Strategies
Four Mitigation Strategies
Choose by scenario; combine for best results
First choice
Prompt Engineering
Role + constraints + examples + CoT. Lowest cost—try this first.
Limitation: Cannot inject new knowledge the model doesn't already have.
Limitation: Cannot inject new knowledge the model doesn't already have.
Private knowledge / real-time data
RAG (Retrieval-Augmented Generation)
Chunk → Embedding → vector search → inject into Prompt. Not "teaching the model"—it's "handing the model a cheat sheet at runtime."
Cost: latency + retrieval quality + maintenance.
Cost: latency + retrieval quality + maintenance.
Adjust output diversity
Temperature / Top-P
T=0 deterministic, T=1 normal sampling, T>1 more random.
Misconception: Higher ≠ smarter; lower ≠ dumber. Controls randomness, not intelligence.
Misconception: Higher ≠ smarter; lower ≠ dumber. Controls randomness, not intelligence.
High-risk / strict compliance scenarios
HITL Human Review
AI generates → human reviews → user sees. Essential safety net during cold-start.
Cost: labor + response latency.
Cost: labor + response latency.
4 · AI PM Mental Model
AI PM Mental Model
Take these judgments with you to make better product decisions
❌"Raising Temperature makes the model smarter"
→Just more random—not more intelligent
❌"RAG = teaching the model documents"
→Just injecting context temporarily at runtime
❌"The model is calling an API"
→The model only outputs formatted text
❌"Retraining will fix errors"
→Try Prompt first—it's 100× cheaper
Prompt is enough
Knowledge is in training data; formatting / style / tone issues
Need RAG
Private knowledge base, real-time data, content after knowledge cutoff
Need fine-tuning
Fixed domain-specific style; reasoning paradigm needs to change
Need HITL
High-risk outputs, brand reputation, medical/legal compliance requirements
Take this with you
At its core, a large language model is a massive probabilistic prediction machine. Understand its limitations—then you can correctly leverage its strengths.