Cost Optimization

Dynamic Timestamps: The Most Expensive System Prompt

One small design mistake can completely invalidate your KV Cache and double the cost of every conversation: putting a dynamic timestamp in your System Prompt.

Wrong vs. Right Design
❌ Wrong Design
System: You are an assistant. Current time: 2026-04-10 14:35:22 ...
0%
Cache Hit Rate
+100%
Extra Cost
✅ Correct Design
System: You are an assistant. (no timestamp) User: [2026-04-10] Please help me...
~95%
Cache Hit Rate
-70%
Cost Savings
Live Comparison Simulation
Live Comparison --:--:-- One request per second
❌ Dynamic Timestamp (second-precision)
System: You are an assistant.
Current time: --:--:--
...
⏳ Waiting for request...
Cache Hit Rate0%
✅ Static System Prompt
System: You are an assistant.
(no timestamp)
User: [--] ...
⏳ Waiting for request...
Cache Hit Rate0%
0
Total Requests
0
Left Cache Miss
0
Right Cache Hit
—
Right Cost Savings
Once started, one request fires per second — watch the KV Cache hit difference between the two sides
Other Common Cache Killers
Other common KV Cache killers:
Second-precision timestamps Random Session IDs User ID prefixes A/B test variables Random emoji prefixes Dynamic ad copy
Anything that makes the System Prompt different each time will completely invalidate the cache.
🔒 Rule: System Prompt = fixed prefix. Put dynamic content (time, user info, random content) in the User message — never in the System Prompt.