Fundamentals

Training vs. Inference: Two Completely Different Processes

Many people assume AI learns something new with every response. In reality, that's completely not the case.

Comparison
Think of a student taking an exam: Training = years of preparation, drilling problems, memorizing knowledge  →  Inference = sitting the exam, parameters frozen, no longer learning anything new
Training
Adjusting billions of parameters on massive datasets so the model memorizes world knowledge.
FrequencyOne-time
Compute1K–10K GPUs
ParametersContinuously updated
CostHundreds of millions $
DurationWeeks to months
Inference
Uses fixed parameters to generate a response to your input. Parameters remain completely unchanged with every conversation.
FrequencyEvery conversation
Compute1–a few GPUs
ParametersCompletely frozen
BillingBilled per Token
Duration1–10 sec/call
Interactive Experiment: Does What You Say Change the Parameters?
Context Window
AI
Hello! How can I help you?
Model Parameters (Illustration)
🔒 Parameters Completely Frozen
Conversations do not change any weights
What you say to the AI does not modify any parameters. If you tell it "your name is Alex" today, it won't remember tomorrow when you start a new conversation — because the conversation only exists in the context window. Conversation is not learning.