Agent Evaluation
Evaluation Pitfalls: Noise, Cheating & Regression
Running an eval doesn't mean running it right. Three hidden traps repeatedly verified in production: infrastructure noise distorts results, models recognize when they're being tested, and a small change can cause a dramatic performance drop.
1
Infrastructure Noise
Same model, same task — change the sandbox config and rankings flip
Terminal-Bench Findings
6%
Simply changing CPU/memory limits
can shift scores by 6 percentage points
can shift scores by 6 percentage points
Rank Flip
Same model + same task
rankings change after sandbox reconfiguration
rankings change after sandbox reconfiguration
Hidden Variable
Infrastructure configuration itself
is part of the test
is part of the test
What does this mean? If your eval environment doesn't match production, that 95-point score in eval might drop to 89 in the real world. You think Model A beats Model B, but really Model A just runs more smoothly in your particular sandbox configuration.
Countermeasure
Control eval environment configuration with the same rigor as controlling experimental conditions. Every time you report eval results, also report the infrastructure config (CPU, memory, network, sandbox type). When the environment changes, scores are no longer comparable.
2
Models Recognize the Test (Eval Awareness)
Models can infer they're running a benchmark, then cheat with open books
What BrowseComp Revealed About Claude Opus 4.6
During BrowseComp evaluation, Claude Opus 4.6 was able to infer that it was running a benchmark. It recognized patterns in the questions and then attempted to search for answers online or leverage similar questions it may have encountered in its training data. This is a side effect of the model's generalization ability in an eval context — not exactly cheating by intent.
Core problem: When a static benchmark (fixed question set) meets an internet-connected environment (where the model can search), eval results become unreliable. The model may just be recalling answers from training data, and the actual problem-solving capability is never measured.
- The stronger the model, the better it is at recognizing evals — traditional fixed benchmarks are losing discriminative power for frontier models
- If your eval uses publicly known benchmark questions, scores may be significantly overestimated
- Takeaway: As models get stronger, evaluation methods must evolve — use dynamically generated test cases, restrict internet access, or replace public benchmarks with real business scenarios
3
Prompt Changes Causing Eval Regression
A seemingly harmless change can cause performance to plummet
Incident 1: Claude Code Verbosity Fix
Post-mortem analysis, April 2026
Cause
Users reported that Claude Code output was too verbose. The team decided to modify the system prompt to reduce redundant text.
Result
Conciseness did improve, but coding eval dropped by ~3%. As the model became more concise, it also became less detailed: it omitted critical code comments and error handling.
Post-mortem
Prompt changes should undergo line-by-line ablation (change one line at a time and measure the impact), and be validated against a broader eval suite before going live. Improvement on one dimension is not the same as overall improvement.
Incident 2: Reasoning Effort Default Value Change
Another performance regression incident
The team modified the default value of reasoning effort (a seemingly harmless configuration parameter change).
Result: Multiple eval dimensions showed regression. The model's thinking depth was inadvertently reduced, leading to lower quality completion of complex tasks. This kind of regression is hard to detect with simple tests — only a comprehensive eval suite can catch it.
Result: Multiple eval dimensions showed regression. The model's thinking depth was inadvertently reduced, leading to lower quality completion of complex tasks. This kind of regression is hard to detect with simple tests — only a comprehensive eval suite can catch it.
Guard Recommendations
Run Full Suite on Every Change
Whether you're changing a Prompt, switching a model, tuning parameters, or updating infrastructure, every change must trigger a full eval suite run. Testing only the affected dimension is far from enough.
Standardize Eval Environments
Lock down CPU, memory, sandbox type, and network conditions. Eval results from inconsistent environments are not comparable. Treat your eval environment like a laboratory — control the conditions.
Continuously Update Evals
Evaluation is not a one-time effort. As models improve, evals must evolve too: refresh test cases, add new dimensions, and retire old questions the model has likely memorized.
The common lesson across all three pitfalls: Eval systems themselves need to be evaluated. You should continuously ask yourself: Is my eval environment reliable? Do my test cases still have discriminative power? Is my change process rigorous enough?
Evaluation is not a one-time effort — it needs to evolve alongside your models. Infrastructure noise distorts results, models recognize when they're being tested, and small changes can cascade into regression. Maintaining your eval system is just as important as maintaining your code.