PART 5 · Harness & Self-Improvement

Workflow Design: From Manual to Automated Search

The design space for workflows is enormous. We should use algorithms to search it — manual design alone is far from enough.

Workflow Design & Automated Search

When the workflow itself becomes the search space, design shifts from art to engineering

From fully hand-crafted pipeline design to using MCTS to automatically discover optimal workflows, the design paradigm for Agent systems is undergoing a fundamental shift. Code is the universal language for defining a Harness.
Hand-Designed Workflows
💬 Plain talk: A workflow is like a task checklist for the AI. Like onboarding a new employee, you have to tell them "do A first, then B, and if something goes wrong check with C." These checklists used to be written entirely by hand, and the two examples below represent the most powerful hand-crafted checklists ever built.
Lu et al. 2026
AI Scientist
Building a complete scientific research automation pipeline
Core idea: Hand the entire scientific research process to an Agent system to complete automatically: execute a full research cycle from scratch.

Pipeline:
Key innovation: Every step is LLM-driven, forming an end-to-end automated research system. The review step introduces LLM-as-judge for quality control.
AI Scientist Pipeline
The complete AI Scientist pipeline: from research ideation to paper writing and peer review, fully automated by Agents. (Lu et al. 2026)
Kulikov et al. 2026
Autodata
Data Scientist Agent: synthesizing training data at exactly the right difficulty
Core idea: Through multi-role collaboration, automatically synthesize training data with precisely controlled difficulty: solvable by a strong solver, unsolvable by a weak solver.

Role system:
ChallengerQuestion setter
Weak SolverWeaker solver
Strong SolverStronger solver
Verifier / JudgeQuality arbiter
Key innovation: Uses difficulty gap as a data quality signal: only questions that a strong solver can solve but a weak solver cannot are retained, ensuring the synthesized data has maximum value for improving the model.
Autodata System
Autodata's multi-role collaboration: Challenger sets questions, Strong/Weak Solvers attempt them, Verifier arbitrates data quality. (Kulikov et al. 2026)
ADAS: Automated Design of Agentic Systems
💬 Plain talk: Writing checklists by hand is exhausting. What if we could let AI design checklists for AI? That's ADAS: hire a "designer AI" and let it continuously draft new checklists, run them, score them, and keep the good ones. It automates the work that used to require a senior expert mentoring a junior one.
Hu et al. 2025
ADAS — Automated Design of Agentic Systems
Treating Agent design itself as an optimization problem
Core idea: Let a "Meta-Agent" automatically search the Agent design space for the optimal solution, replacing hand-crafted Agent architectures. This is "meta-agent search".

How it works:
1. Meta-Agent generates a high-level description (Agent architecture, tool usage, reasoning strategy)
2. Translates the description into executable code
3. Uses self-refine to check for novelty: ensures it's not simply repeating an existing solution
4. Evaluates on benchmark tasks, keeping the best-performing designs

Key innovation: Uses code as the representation of the Agent design search space, enabling designs to be automatically generated, modified, and evaluated.
ADAS Meta-Agent Search
ADAS Meta-Agent Search: Meta-Agent generates candidate Agent designs → implements as code → evaluates → iteratively optimizes. (Hu et al. 2025)
AFlow: MCTS-Based Workflow Search
💬 Plain talk: ADAS lets AI freestyle and come up with new solutions. AFlow goes further, using the same board game simulation technique (MCTS, Monte Carlo Tree Search) that powers game-playing AIs. Just like AlphaGo simulates hundreds of moves ahead — "if I play here, what will my opponent do?" — AFlow simulates hundreds of workflow variations in its head and digs deeper down whichever path scores highest.
Zhang et al. 2025
AFlow
Representing Agent workflows as graphs and optimizing automatically with MCTS
Core idea: Represent workflows as a directed graph: nodes are LLM action calls, edges are logical operations in code (conditional branches, loops, data passing). Then use Monte Carlo Tree Search (MCTS) to automatically find the optimal workflow in this graph space.

MCTS optimization process:
  1. Initialize the starting workflow as the root node of the search tree
  2. Use soft mixture of score and uniform exploration to select the node to expand (balancing exploitation and exploration)
  3. Let the LLM generate modified workflow variants (add/remove/modify nodes and edges)
  4. Execute and evaluate the new workflow's performance on the target task
  5. If there is improvement, add it back to the search tree as a new candidate
  6. Repeat until the top-k average score stabilizes or the compute budget is exhausted
Key innovation: Transforms the workflow design problem into a tree search problem, letting MCTS's exploration-exploitation balance mechanism automatically discover good workflow structures — no human hand-crafting required.
AFlow Architecture
AFlow's graph representation and MCTS search: workflows are represented as directed graphs of nodes (LLM actions) and edges (logical operations). (Zhang et al. 2025)
AFlow Experimental Results
Method Design approach Search strategy Core advantage
Manual design Human expert iteration None (intuition-based) High interpretability
ADAS Meta-Agent + code Self-refine Automated design
AFlow Graph representation + MCTS Monte Carlo Tree Search Systematic search + stable convergence
AFlow Experimental Results
AFlow performance on QA, code generation, and mathematical reasoning tasks: systematic search outperforms manual design and ADAS. (Zhang et al. 2025)
Step-by-Step: AFlow's MCTS Search Process
Ready
1
Initialize
W₀: Plan → Execute Score 52
Start with the simplest single-step workflow. No reflection, no verification, no parallelism.
2
Expand: LLM Proposes Variants
W₁: +Reflect61 W₂: +Verify58 W₃: +Decompose55
Let the LLM generate three modification proposals based on W₀, adding reflection, verification, and task decomposition respectively.
3
Evaluate & Select Parent Node
W₁: 61 ← Selected
W₁ has the highest score (61 > 58 > 55), so it's selected as the parent node for the next round of expansion.
4
Expand Again
W₄: Reflect×265 W₅: Reflect+Verify78 🏆
Expand two variants from W₁: double reflection vs. reflection + verification combo. The latter wins by a wide margin!
5
Prune & Converge
W₃: 55 ✗ W₆: 56 ✗ W₅: 78 ✓ Optimal
Workflows below the threshold are pruned. W₅ (Plan → Execute → Reflect → Verify → Output) is confirmed as the optimal solution found. 50% better than manual design!

Core Insights