Prompt Harness

The Subtle Relationship Between Prompts and Caching

The rule of KV Cache is extremely simple: if the prefix is identical, it's a hit; differ by even one character and everything is recomputed. So injecting any dynamic content into the System Prompt may make the cache permanently miss.

Changed fingerprint = Cache invalidated

Every request, the system computes a fingerprint (hash) of the System Prompt

System: You are an assistant. Current time: 14:35:22
→
a3f7c2
MISS
System: You are an assistant. Current time: 14:35:23
→
b8e1d9
MISS
System: You are an assistant. Current time: 14:35:24
→
c4a2f6
MISS

Different time each second → different fingerprint → cache never hits → recomputed every time → costs double.
This isn't just a timestamp issue: any dynamic content injected into the System Prompt has the same effect.

Skill injection method also affects caching
Inject Skill into System
[System] You are an AI assistant... [Current Skill: Content Creation] Trigger: Help me write... Steps: 1. Confirm requirements... --- [User] Help me write an article

Switching Skills changes System → cache invalidated

~20%
Skill appended as a separate message
[System] You are an AI assistant... --- [Assistant/Skill message] Current Skill: Content Creation Steps: 1. Confirm requirements... --- [User] Help me write an article

System never changes → prefix cache always hits

~90%
Simulation: Cache Performance Across 10 Requests

Cache Hit Simulator

Assume 3 out of 10 requests switch to a different Skill. Observe the cache difference between the two injection approaches:

Hit rate：—
Recall Lesson 8-3: We previously discussed how dynamic timestamps are the most expensive anti-pattern. The conclusion of this lesson is broader: it's not just timestamps — any content dynamically injected into the System Prompt is a cache killer. Skills, user IDs, session markers… anything that changes should never appear in the prefix.
The rule of KV Cache is simple: same prefix = hit. So don't touch the prefix. Put everything that might change (Skills, context, timestamps) after the System Prompt, and keep the prefix permanently stable.