Trust Calibration: The Best Users Are Half-Skeptical
The first four lessons scored points for experience. This one hits the brakes: for trust as a metric, the target was never a perfect score. AI will err—that’s an unfixable physical fact—so a healthy user looks like this: use it freely where it’s safe to trust, keep a wary eye where you should check. Drift either way and you pay tuition. This lesson starts with two real tuition bills, then hands you three calibration tools you can tweak yourself.
⚠ Overtrust: treating hallucination as truth
2023, New York—Mata v. Avianca: a practicing lawyer used ChatGPT to find case law and pasted 6 fabricated cases that don’t exist straight into court filings. After the judge checked each one, the lawyer was sanctioned and made global news. Similar accidents kept coming: AI output is fluent, confident, and well formatted—every surface cue nudges the judgment “this looks solid”, and those cues have nothing to do with whether the content is true.
⚠ Undertrust: AI becomes an expensive paperweight
The crash at the other end is quieter: a company buys AI tools, an employee hits an error once, and from then on every line of output gets sentence-by-sentence review. Review costs more than writing it yourself, so people stop using it. Procurement keeps paying; efficiency never rises. Undertrust doesn’t make the news—it only shows up in the internal postmortem titled “AI tool active usage: 8%.”
Here’s the judgment mantra first: watch “risk × verifiability”, not “how powerful AI is.” The same full adoption is calibrated on a weekly report and overtrust in court. Six scenarios—you’re the calibrator.
Calibration tool one: source citations—the kind you can open. Users who want to verify jump to the original in one click, which reins in overtrust; users too lazy to verify still see “there’s a source” and get reasonable confidence, which patches undertrust. The win-win requires citations that are real and clickable: a decorative fake citation exposed once is ten times worse than no citation. The AI answer below hangs three superscripts—open each one and compare with the source text.
Tool two: confidence. Part One already covered it—the model doesn’t know what it doesn’t know, so self-reported certainty is unreliable. But the product layer has honest proxy confidence signals: how many docs retrieval hit, how relevant they are, whether sources agree, whether the knowledge cutoff covers the question. Below is the same medication answer; three switch groups map to three product decisions—flip them and watch how the answer on the right and the user’s trust calibration change.
Tool three does one job: that tiny line—“AI may err; please verify important information”—is anyone actually reading it? Psychology’s answer is banner blindness: Benway & Lane (1998) found with eye-tracking that elements constantly shown in a fixed spot get filtered out by the brain as background texture. Tap through five answers below and watch that line disappear with your own eyes.
✅ What this lesson wants to share
- Set the trust target at calibration: full trust causes accidents, no trust wastes money—pull users toward the diagonal
- Make citations clickable: swap “trust me” for “you can verify,” and watch for decorative fake citations biting back
- Say confidence with proxy signals: retrieval hit count, relevance, source agreement—more honest than the model’s self-reported certainty
- Show warnings conditionally: a constant disclaimer goes invisible in three days; a low-confidence popup with a reason is what gets read