PART 5 · Harness & Self-Improvement

Letting the Harness Improve Itself

If an LLM can optimize the code that runs the Agent, it gains access to a design space far larger than hand-written Prompts.

STOP: Self-Taught Optimizer
💬 Plain talk: Imagine a craftsman who sharpens his own knife. The ordinary approach is to use the knife to cut vegetables (using a tool to solve a problem); STOP's approach is to first master the skill of sharpening, so the knife naturally gets sharper over time. Even better, the sharpening skill itself can be sharpened: use the improved method to improve the "improving method" itself—layer upon layer, growing stronger with each iteration.
Core Idea: Don't improve the solution—improve the improver
STOP (Zelikman et al. 2023) is an early example of recursive scaffold improvement. It does not directly improve a solution s. It continuously improves the improver I that produces better solutions.

The seed improver I₀ takes three inputs: an initial solution s, a utility function u, and a black-box language model M, and returns an improved solution s'.

Key insight: the improver itself is text (a prompt or code), so the same improvement logic can be applied to the improver itself.
// STOP recursive update formula
I_t = I_{t-1}(û, I_{t-1}; M)

// Where:
// I_{t-1} — current improver (prompt/code in text form)
// û — meta-utility: function measuring improver quality
// M — black-box language model
// Output I_t — a better improver
💬 What does this formula mean? Don't be intimidated by the notation. It says just one thing: let today's self upgrade yesterday's self. I is that upgrade method; the formula says: apply the old upgrade method to itself, and produce a new upgrade method. Just like using your 3.0 learning method to derive your 4.0 learning method.
STOP algorithm flow
STOP algorithm flow: the improver recursively acts on itself, generating increasingly better improvement strategies. (Zelikman et al. 2023)
STOP's Findings
Automatically Discovered Strategies

The improved improver automatically discovered classic optimization strategies including genetic algorithms, decomposition-based improvement, multi-arm Prompt Bandit, simulated annealing, and Beam Search—all without human pre-specification.

Base Model Threshold

GPT-4 can improve consistently, but GPT-3.5 and Mixtral actually regress. The recursive structure alone is insufficient: the base model must be strong enough to support meta-level optimization.

Optimization patterns discovered by STOP
STOP performance across different models: only sufficiently capable base models can sustain continuous improvement through recursive improvement. (Zelikman et al. 2023)

Warning: Recursion ≠ Guaranteed Improvement

The recursive structure provides the possibility of improvement, but does not guarantee convergence. Weak models lack sufficient programming intuition at the meta level, and instead amplify noise: results regress rather than improve. This reminds us that the safety of self-improving systems must be grounded in an accurate assessment of base model capability.

Self-Harness: Agent Improves Its Own Harness
💬 Plain talk: Think of an employee doing their own performance review: ① dig out all the cases that went wrong and identify recurring pitfalls (Weakness Mining); ② revise their own work manual accordingly (Harness Proposal); ③ run the new manual for a trial period to confirm existing skills haven't regressed and no new bugs appeared, then officially adopt it (Validation). The entire process requires no manager intervention.
Propose → Evaluate → Accept Loop
Self-Harness (Zhang et al. 2026) lets an LLM Agent improve its own Harness configuration through a three-phase loop. Unlike STOP's pure text improver, Self-Harness directly operates on the Agent's runtime orchestration system—including system prompt, tool scheduling strategy, validation rules, and more.
1

Weakness Mining

Clusters failure trajectories into verifier-grounded failure patterns. Each failure record must include: terminal-verifier-level cause + causal state of related Agent behavior + abstract Agent mechanisms exposed by the trajectory.

→
2

Harness Proposal

Proposes bounded Harness edits based on mined failure patterns. The model receives: the editable surface, failure pattern summaries, records of passing behavior, and a history of previously attempted edits. Priority is given to addressable recurring error patterns.

→
3

Proposal Validation

Validates candidate edits using held-in and held-out datasets. Only accepts edits without regression, ensuring improvements don't sacrifice existing capabilities.

Self-Harness flow diagram
Self-Harness three-phase loop: learn from failures, propose bounded edits, accept after validation. (Zhang et al. 2026)
Experimental Validation
Experiments were conducted on Terminal-Bench-2 with MiniMax M2.5, Qwen3.5-35B-A3B, and GLM-5. Self-Harness learned model-specific harness instructions for each model: the same framework produced different optimization paths for different base models, confirming that Harness improvement is indeed context-sensitive.
Key Insights

Harness Design = Executable Search Space

Once Harness design is formalized as an executable search space (editable prompts, policy configurations, tool orchestration code), a capable coding Agent can exploit the same design space that human engineers use: automating search, proposal, and validation without relying on manual debugging case by case. This opens up improvement possibilities far larger than hand-written prompts.

Safety Boundaries Are Indispensable

If a program is allowed to edit OS-level configurations, abstraction boundaries will be violated. The editable surface of a self-improving system must be carefully designed: permission controls and security layers must sit outside the improvement loop, guaranteed by humans or tamper-proof oversight mechanisms. Self-improvement without boundaries is self-improvement out of control.