Chapter Zero · Beginner FAQ

What Is a Token? Why Does AI Charge by It?

The bill says "billed by token," the news keeps talking about "a million tokens" — what even is a token? This page lets you slice a sentence open and see for yourself.

One-sentence answer

A token is the smallest unit AI uses to read and write text — roughly a small chunk of a word. Every token it reads and every token it writes burns compute, so you're charged per token. Same logic as a taxi charging by distance: the farther you go, the more you pay.

Slice a sentence yourself

A few ready-made sentences are below. Click one and try. You'll see it get cut into colored chips — each chip is one token — and the counter and bill below will jump in real time.

Click any sentence above to see it sliced into tokens
0
Characters
0
Tokens
0 li
Read cost for this sentence (demo)
The split is the real result from the GPT-4o family's tokenizer (o200k). Other models will cut differently. Notice it doesn't follow human intuition: 「今天天气」 gets cut into 「今 | 天天 | 气」. The bill is estimated at a demo rate of "about 2 li per 1,000 tokens" (1 li = 0.001 yuan) — just to give a feel for the scale.
Why charge by token

A taxi charges by distance because every kilometer really burns fuel. AI is the same: every token it reads or writes, a GPU in a data center does a round of computation and uses electricity. Charging by count is the closest match to real cost. And note: the bill counts both ends:

What you send (input)

Every character you type, every document you paste — it has to read them token by token. Reading costs money too.

What it writes back (output)

Every character it writes is also computed token by token. Writing costs money too — and the unit price is usually higher, because writing is harder than reading.

So pasting a long chunk of unused material for it to read, or letting it ramble in circles, burns money on both ends. The prompt engineering page covered this: fluff in the prompt gets magnified, token by token, into the bill.
Does Chinese cost more, or English?

Use the tokenizer above to compare a Chinese sentence and an English one: Chinese often takes one token for every one or two characters, while a fairly long English word often takes just one. So to say the same thing, Chinese often burns more tokens. It has to do with how the model "reads": it's seen more English and cuts it more fluently; Chinese characters get sliced finer.

Common misconception: character count ≠ token count. Numbers, symbols, and rare words get sliced especially fine — a code like "#2049" takes three tokens; 「今天天气真不错」 is seven characters but only five tokens. Don't estimate the bill from character count — the scale will be way off.

✅ What this page wants to share with you