Agent Design Patterns
From Prompt Engineering to Context Engineering
As we move from single-turn conversations to multi-step Agents, optimizing prompts alone is no longer enough. The real challenge is: how do you curate every Token sent to the model at each inference step.
Conceptual Evolution
Past
Prompt Engineering
Optimizing how prompts are written: wording, structure, Few-shot examples
Now
Context Engineering
Curating all Tokens sent to the model at each inference step: System Prompt, tool definitions, MCP descriptions, conversation history, externally retrieved data...
Prompt engineering focuses on how to write instructions, while context engineering addresses a larger question: what goes into the model's input window, how it's arranged, and how much of it there is. When your system has a System Prompt, tool descriptions, message history, RAG retrieval results, and user preferences — these can fill up most of the context window. Managing those Tokens is what context engineering is all about.
What's Inside the Context Window
System Prompt — Role definition, rules, constraints
Tool Definitions — Tool names, parameters, descriptions
Conversation History — Multi-turn dialogue history
Retrieved Data — RAG retrieval results, file contents
User State — User preferences, session state, environment info
All of the above combined = the complete information the model sees at each inference step
Why Context Engineering Matters
Context Rot
The longer the context, the lower the model's retrieval accuracy. Key information gets buried in a sea of Tokens.
Limited Attention Budget
Every Token consumes part of the model's attention budget. Irrelevant Tokens take up space = useful information gets diluted.
n-squared Complexity
n Tokens produce n×n attention relationships. Double the context, and compute cost quadruples.
Try It: Context Rot Simulator
1K Tokens — a short conversation
Attention focused, retrieval accurate
Retrieval Accuracy
Attention Density
High attention Low attention
Understanding the Cost of Attention
In the Transformer self-attention mechanism, each Token must compute its relationship with every other Token:
Attention Complexity = O(n^2)
This means: extending the context from 50K to 100K Tokens causes attention compute cost to grow 4×, far more than doubling. Context is not free: every extra irrelevant Token wastes attention that could go to other Tokens.
Three Principles for Efficient Context
The Right Height for Your System Prompt
Too vague ("you are a helpful assistant") = the model lacks direction, outputs generic responses.
Too specific (listing 50 edge cases) = the model is over-constrained and can't handle novel situations.
Best practice: provide a clear role and core principles (5–10 items), then trust the model to make its own judgments within that framework. Like a good manager: give direction, not step-by-step instructions.
Too specific (listing 50 edge cases) = the model is over-constrained and can't handle novel situations.
Best practice: provide a clear role and core principles (5–10 items), then trust the model to make its own judgments within that framework. Like a good manager: give direction, not step-by-step instructions.
Finding the sweet spot in the middle is the goal for System Prompt length
Keep Your Toolset Lean
Validated in production: if a human can't tell which tool to use, neither can the AI.
Giving an Agent 10 tools with overlapping capabilities and vague descriptions is worse than 5 tools with clear responsibilities and precise naming. Each tool's description should read like good API documentation — the caller (the model) should know immediately when to use it and how.
Giving an Agent 10 tools with overlapping capabilities and vague descriptions is worse than 5 tools with clear responsibilities and precise naming. Each tool's description should read like good API documentation — the caller (the model) should know immediately when to use it and how.
Curate Few-shot Examples — Don't Pile Them Up
Few-shot examples have the highest ROI of anything in the context, but only if you choose the right ones.
Right approach: curate 2–3 highly representative examples that cover the most common input patterns.
Wrong approach: pile on 10+ edge-case examples, which wastes Tokens and causes the model to over-focus on edge cases while neglecting the main use case.
Right approach: curate 2–3 highly representative examples that cover the most common input patterns.
Wrong approach: pile on 10+ edge-case examples, which wastes Tokens and causes the model to over-focus on edge cases while neglecting the main use case.
Context is a scarce resource. Your goal is to find the minimal high-signal Token set. Every Token must contribute to the model's reasoning — the "throw everything in" approach doesn't work. Edit your context the way an editor refines an article: every unnecessary word is noise.