Fundamentals
Base Model: The Token-by-Token Prediction Machine
The raw model you get after training does exactly one thing.
After Training
What you get is a
relentless Token-by-Token prediction machine
It knows exactly one thing:
Given all preceding Tokens, predict the single most probable next Token.
Given all preceding Tokens, predict the single most probable next Token.
Context Tokens
→
Probability Distribution
→
Sampling
→
New Token
→
Repeat
It does NOT
Understand questions · Reason through answers · Look up knowledge
It only sees probabilities, only emits Tokens.
It only sees probabilities, only emits Tokens.
What this means
Feed it 「紫霞捧着月光宝盒，轻声问：哥哥」（"Zixia cradled the Moonlight Treasure Box and softly asked: Brother…"）,
it will predict the statistically most likely next word,
but it does not know what it is writing.
it will predict the statistically most likely next word,
but it does not know what it is writing.
Token Generation Demo
Context (Preceding Tokens)
紫霞
→
捧着
→
月光
→
宝盒
→
，
→
轻声
→
问
→
：
→
哥哥
This is everything the Base model does: keep appending the highest-probability word to the sequence until a stop token is generated.
No intent, no memory, no common-sense reasoning — only probabilities, only Tokens.
Yet this simple loop is the underlying engine behind every capability of large language models.