Chapter Zero · Beginner FAQ

Why Can't Some AIs "See" Pictures?

You've probably run into this: you send an AI a photo and it replies "sorry, please describe it in words." You've also seen ones that can read an entire recipe out of a photo. Both are AI — what's the difference?

One-sentence answer

Because "speaking" and "seeing" are two independent skills. A language model natively understands only text; to make it see, you have to fit it with an extra pair of "eyes." The ones that have them are called multimodal models — the ones that don't can only apologize to your photo.

A Little Experiment · Send the Same Cat Photo to Two Models
Model A · Text-only model (no eyes fitted)
🖼️ cat-photo.jpg
"Sorry, I can't view images. If you describe what's in the picture, I'll be happy to help analyze it."
Model B · Multimodal model (eyes fitted)
🖼️ cat-photo.jpg
"An orange cat is lounging on a bay window in the sun, right front paw resting on a ball of yarn. Judging by the physique… this one might need to watch its diet 😄"
How It Works · How the "Eyes" Get Fitted
✂️

Step 1: chop the image up

The "eyes" (a vision encoder) slice the image into many small squares, translating each into "words" the model understands — a picture becomes a special kind of "text."

🔤

Step 2: continue it like a sentence

Once translated, an image is no different from a passage of text to the model, and it "finishes the sentence" as usual. So under the hood, image understanding is still that same sentence-finishing machine.

🏋️

Why not fit eyes on everything?

Fitting eyes means retraining on massive amounts of paired image-and-text data — expensive, and the model gets bigger and pricier. Many use cases never need vision, and text-only models are faster and cheaper.

While we're here, let's clear up a common mix-up: "understanding an image" and "creating an image" are two different things. A model that can see pictures can't necessarily generate them — the former is the "eyes," the latter is an entirely different "painter" (the next page explains why that one is so expensive). If the AI app you use can both see and draw, it's quietly juggling several different models behind the scenes.

✅ What this page wants to share with you