AI Product Psychology

Mental Models: Users Brought the Wrong Manual

Interaction design has an old idea called the mental model: users arrive with assumptions about “how this thing should work.” Wrong assumptions mean wrong actions—then the one-star review lands on the product, and the bill lands on you. AI is the first thing in human history that “looks like an old product but runs on a different core,” so mismatches hit especially hard. This lesson first names four wrong manuals, then lets you play three correction toys yourself.

Four wrong manuals · wholesale source of bad reviews

The chat box looks like a search box, answers look like an encyclopedia, the UI looks like a patient support agent. Users match four old manuals by appearance—and each one manufactures a precise class of bad review.

Wrong manual ①
Used as a search engine
ExpectResults have sources; information is real-time
RealityAnswers from memory, knowledge has a cutoff, no links. Chapter Zero already said “search gives you shelves; AI gives you conclusions”—but users never took that class
Review“The links you gave me don’t open” · “Can’t even look up today’s news”
Wrong manual ②
Used as a database
ExpectAsk and look up; if it’s missing, say so
RealityA probability-completion machine—when it can’t find something, it invents a plausible answer
Review“I asked for our Q3 revenue and it just blurted a number—scared me to death”
Wrong manual ③
Used as a learning apprentice
ExpectTell it once and it remembers; the more you use it, the more it gets you
RealityParameters are frozen; a new session means amnesia—“gets me better over time” is just notes stuffed back into context
Review“Told it eight hundred times not to use tables—turns around and forgets. Attitude problem”
Wrong manual ④
Used as a calculator
ExpectMath is supposed to be correct, period
RealityLanguage models emit numbers by token probability; long formulas and multi-line sums often fail. The right move is to have it write code to compute
Review“Can’t even sum 37 numbers—and this is ‘intelligent’?”
Spot-the-difference drill · name the manual in their hands

Four dialogues—the accident has already happened. First decide which manual the user is holding, then read the explain: each one includes a product fix. Diagnosis is only the start.

Four dialogues, four misdiagnosis scenes 0 / 4 judged
The clues are in the chat: see what the user wants, then what the AI actually gave.
Correction move 1 · turn the empty state into a classroom

You can’t fix a mental model with a help doc—nobody reads it. You teach in the cracks of the usage path, and the first scene is the blank first screen of a new session: the user hasn’t typed a word yet, the manual isn’t open—what this screen says decides which book they pick up. Three first-screen schemes—flip them yourself, and watch the UI on the right and first-question quality below change.

Empty-state designer Flip it
New-user first-question quality
24
Your AI assistant · New session
Message the AI…
Correction move 2 · let users see “Remembered” with their own eyes

The apprentice-manual review phrase is “I told you and you didn’t listen.” Both products below agreed to the user’s request—the difference is how they agreed. Tap the version you think can kill that review line.

“Remembered” visibility duel Pick one
Both versions’ users said the same sentence. Scroll down to see what happened three days later.
Version A
From now on don’t reply with tables—I paste straight into WeChat groups.
Got it.
Three days later · new session
Help me compare these three projectors.
Here’s a comparison table:
ModelBrightnessPrice
X1800 lumens2299
X21200 lumens3499
Version B ✓ Saved preference: avoid tables
From now on don’t reply with tables—I paste straight into WeChat groups.
Got it—I’ll use bullet lists instead.
Three days later · new session
Help me compare these three projectors.
Per your preference · no tables · X2 is brightest—clear even in a daytime living room; · X1 is cheapest, wins on light and easy storage; · X3 has the smoothest system—pick it for binge-watching.
Correction move 3 · state the boundary before the mistake

Time-sensitive questions are a hot zone for the search-engine manual—spot-the-difference dialogue one crashed exactly that way. Both AIs say the same boundary text, word for word; the only difference is when they say it. Pick the version that better protects the mental model.

Boundary-first duel Pick one
Same question, same knowledge cutoff, two timings.
Version A · explain after the miss
What’s today’s USD/CNY exchange rate?
USD/CNY is about 7.24.
User checks the bank app: actual 7.09
That’s wrong—why such a gap?
Sorry. My knowledge cuts off at June 2025—I can’t get live rates. What I gave was an old value from training data.
Version B · declare before answering
What’s today’s USD/CNY exchange rate?
This is a live-data question: my knowledge cuts off at June 2025—for this kind of ask, turn on web search. Web searchOn
Looked it up live: today’s USD/CNY midpoint is 7.09 (source: China Foreign Exchange Trade System, updated 10:15).
The calculator manual gets the same prescription: sums, stats, date math—route them to a code interpreter, and label the UI “computed with code.” Error rates drop, and users also see the point: the trustworthy path is calling tools; the barehanded arithmetic was never meant to be trusted.
Sources and further reading: The mental-model idea comes from Craik (1943); Norman brought it into product design in The Design of Everyday Things: users act on their own model—wrong model, wrong action. Memory visibility maps to the first of Nielsen’s ten usability heuristics—“visibility of system status”; putting examples in the empty state also stacks Cialdini’s social proof from Influence: The Psychology of Persuasion: others ask this way, so I will too. Evidence that a first miss triggers abandonment is in Lesson 6 on algorithm aversion (Dietvorst et al., 2015).

✅ What this lesson wants to share