Chapter Zero · What Exactly Is AI

It Spouts Nonsense with a Straight Face

From the last two lessons you know: it finishes sentences, and it does so from "digested memory". Now comes the most important lesson of all — sounding right is not the same as being right. Each of the three AI answers below hides one fabricated sentence. Go ahead — catch it yourself.

Spot-the-fake · One sentence in each answer is made up — click it
Why does this happen? Back to "finishing sentences"
Suppose it's finishing the sentence: "Yu Hua's notable works also include…" — and it actually doesn't remember clearly. The candidates in front of it right now:
Smooth · easily picked "the novel Cries in the Drizzle." — a real title. But if that book isn't in its memory, it can just as smoothly produce a title that doesn't exist yet sounds exactly right, like The Age of Mistakes by the River.
Awkward · rarely picked "Er, I don't quite remember this one — I'd rather not guess." — across the hundreds of millions of books it has read, authors almost never write like this, so this continuation is inherently "not how text usually sounds" and rarely gets picked.
See it now? It picks "whichever sentence looks most like it belongs here", not "whichever sentence is true". In human writing, fluent confident statements vastly outnumber "I don't know" — so a smooth fabrication naturally beats an honest hesitation. It hasn't turned dishonest; this is a side effect of the sentence-finishing mechanism — the technical term is "hallucination".
The most crucial point: it doesn't know that it doesn't know. A person telling a lie knows it inside; the AI doesn't — to it, being right and being wrong are the same single action: pick something that fits and attach it. Which also means it has no way to warn you "I made this sentence up".
Three little habits · Usable starting today
🔍

Habit one: concrete facts are worth verifying before you use them

When you see numbers, dates, names, book titles, legal provisions, medication doses — things with "one correct answer" — look up the original source before relying on them. AI saves you understanding time; it can't save you verification time.

💬

Habit two: ask one more question — "Are you sure? What's the source?"

This follow-up often makes it correct itself ("Apologies, my earlier statement wasn't accurate…"). But note: the source it gives may itself be fabricated — inventing a plausible-looking paper title costs a sentence-finishing machine nothing. Open and verify sources yourself.

⚖️

Habit three: the bigger the decision, the more a second source is worth

For anything involving health, money, or the law, treat AI's answer as "a well-informed friend's opinion, for reference" — a friend's word has value, but you wouldn't take medication, sign a contract, or go to court on a friend's say-so alone.

✅ What this lesson wants to share with you