Foundational Principles
AI's Food: Training Data
To teach an AI to speak, you first have to feed it trillions of words. It has read the vast majority of everything humanity has ever written.
Data Scale & Composition
~1B
Tokens · Training data scale for a very small model
What a model has read is the ceiling of what it can say.
The quality and diversity of training data set the upper bound of a model's worldview.
The quality and diversity of training data set the upper bound of a model's worldview.
Drag the slider · Feel the capability gap across model sizes
270M
Hallucination
Q: Who is Lee Ji-eun (IU)? Please describe her life and key works.