How They Will Test You

Harness & Self-Improvement · 6 Soul-Searching Questions

This chapter covers the most cutting-edge content, and those who test you on it can most easily tell the real from the fake. These 6 questions come from three real scenarios — try answering aloud first, then check the framework.

How to Use This Page
Each question is labeled with who is asking. They are all testing the same knowledge block, but each wants to hear something different.
🎙 Interviewerwants to verify whether you truly understand or are just reciting terms
👔 Bosswants explanations and commitments
🛠 Tech Colleagueprobing whether you are worth trusting
Each question has three layers: What they are testing → Answer framework → Bonus points. For parts you cannot answer, click the course links at the end to review.
Q1Interviewer
"Everyone's been talking about Harness lately — can you explain what it actually is? Why do some people say it's just as important as the model itself?"
🎯 What They Are Testing
This is the opening litmus test for this chapter. It checks whether you can articulate Harness as a clear system concept, or if you only vaguely say "it's just a wrapper" or "just an Agent framework." Someone who truly understands it can explain what it governs and why it determines product success or failure.
🧭 Answer Framework
  1. Define it first: Harness is the runtime system surrounding the base model. It determines how the model thinks and plans, calls tools and acts, perceives and manages context, stores artifacts, and evaluates results. The model provides the intelligence; Harness makes that intelligence work in the real world.
  2. Provide evidence: Successful products like Claude Code, Codex, and Cursor have proven that the Harness layer is as important as the raw model intelligence. A mediocre model with an excellent Harness often outperforms a stronger naked model.
  3. Explain the PM perspective: Swapping models is easy — Harness is the product's true moat. With the same model, the quality of the Harness design can produce user experience differences of several orders of magnitude.
  4. Elevate the insight: Harness is shifting from a supporting engineering role to becoming the optimization target itself. Cutting-edge research lets models improve the Harness surrounding them — this is the real-world path to recursive self-improvement.
⭐ Bonus PointDraw the Prompt Engineering analogy: manual Prompt techniques faded as models got stronger, but the need to specify goals, constraints, context, and evaluation never went away. Many Harness improvements will eventually be internalized into models, but the interface with external context and tools will always exist.
Use these course pages to organize your answer → From Scaffolding to Self-Improving Systems Three Design Patterns for Harness
Q2Interviewer
"If you were to design a production-grade Agent architecture, what design patterns would you follow? Why do the best coding Agents today all look similar?"
🎯 What They Are Testing
They are checking whether you have an architectural vocabulary. A revealing answer just lists tool names and framework names. What they want to hear is the pattern-level abstraction: why these systems converge to similar structures, and what problem each pattern solves.
🧭 Answer Framework
  1. Name the three patterns: workflow automation, filesystem as persistent memory, and sub-Agents with background tasks. These cover 90% of the architectural decisions in today's most capable Agent systems — they are structural requirements with no real optionality.
  2. Pattern one — the loop: An Agent is a goal-oriented loop: Plan → Execute → Observe → Improve → Execute again. Failure is the trigger for self-correction: a failing test or a command error causes the Agent to analyze its trajectory and adjust.
  3. Pattern two — memory: Artifacts from long tasks quickly overflow the context window. The right approach is to persist state to the filesystem and let the Agent read and write as needed. One-sentence principle: context is working memory; filesystem is long-term memory.
  4. Pattern three — parallelism: The parent Agent acts as a process manager: spawning sub-Agents, polling progress, canceling failed branches, and merging results. Sub-Agent outputs must be persisted to files so they can be recovered after interruption.
⭐ Bonus PointArticulate the key tradeoff in sub-Agent parallelism: each sub-Agent works in an isolated sandbox and outputs to explicit file paths; the parent Agent coordinates by polling file state rather than sharing memory. This design dramatically simplifies concurrency control — being able to articulate it signals you have seen real production systems.
Use these course pages to organize your answer → Three Design Patterns for Harness From Scaffolding to Self-Improving Systems
Q3Boss
"That Agent of ours — once the task runs long it starts making mistakes, forgetting everything said earlier. Can this be fixed? How long will it take?"
🎯 What They Are Testing
The boss wants diagnosis + solution + timeline. A revealing answer is "just use a model with a bigger context window" — that shows you haven't understood the root cause. The real problem is in the context management strategy, and only someone who can explain that and provide a phased plan is worth trusting.
🧭 Answer Framework
  1. Diagnose the root cause first: Most likely it is naive append-only context management. Stuffing all tool responses and history into the context fills the window as the task runs long, pushes out early information, and causes output quality to plummet. This is a strategy problem — a bigger context window only delays the symptoms.
  2. Phase 1 solution: Introduce filesystem persistent memory. After each round, write progress, error logs, and intermediate results to files, freeing the context; read selectively on the next round. Context usage goes from continuously growing to constant, enabling stable runs of dozens of rounds.
  3. Phase 2 solution: Introduce ACE-style structured context maintenance. Generator does the work, Reflector reviews and distills insights, Curator organizes insights into itemized playbooks, incrementally merging and deduplicating. Experience gets sharper with use while the context gets leaner.
  4. Timeline commitment: Phase 1 is an engineering refactor — results visible within a week. Phase 2 requires building an evaluation set to verify convergence; report changes in long-task success rates bi-weekly using metrics.
⭐ Bonus PointAdd an industry observation: long-context capability and context engineering are two separate legs. No matter how large the model's window, you still need a management layer to build structured, lean context. Every team building Agents will face this problem — solving it early is a competitive advantage.
Use these course pages to organize your answer → Filesystem as Persistent Memory Context Engineering: From Manual to Autonomous Evolution
Q4Interviewer
"Isn't context engineering just about writing better prompts? I saw a paper saying it can self-evolve now — are you familiar with ACE and MCE?"
🎯 What They Are Testing
They are checking your depth and sense of layering for cutting-edge concepts. Someone memorizing terms will lump ACE and MCE together. Someone who truly understands can articulate an evolution line: the optimization target upgrades layer by layer — from content to mechanism to system code — and what each layer solves.
🧭 Answer Framework
  1. Correct the premise first: Manual Prompt writing is just the starting point. The optimization target follows an evolution line: instruction Prompt → structured context → workflow → Harness code → optimizer code. The stronger the model, the more complex the target it can optimize.
  2. Explain ACE: It optimizes context content. Maintains a structured bullet-point playbook; Generator executes tasks, Reflector distills insights from success and failure trajectories, Curator incrementally merges entries using deterministic logic. Never rewrites in bulk — avoids context collapse. The limitation is that update rules still require manual design.
  3. Explain MCE: Separates "how to manage context" from "what is in context" — two-layer optimization. The inner layer finds the optimal context for a given skill; the outer layer compares different skills to select the optimal mechanism. Both what you remember and how you remember evolve together.
  4. Explain Meta-Harness: One layer deeper — the optimization target is the code itself that determines how information is stored, retrieved, and presented. The Proposer is a coding Agent that outputs a set of Harness candidates on the Pareto frontier. Most general, but each revision requires a full trial run to score, making it the most computationally expensive.
⭐ Bonus PointSummarize the three differences in one sentence each: ACE optimizes the note content, MCE optimizes how notes are taken, Meta-Harness optimizes the source code of the entire workbench. Add the core lesson: once Harness design becomes an executable search space, a capable coding Agent can explore the same design space as a human engineer.
Use these course pages to organize your answer → Context Engineering: From Manual to Autonomous Evolution Workflow Design: From Manual to Automated Search
Q5Tech Colleague
"You want the Agent to modify its own Harness? What if it also modifies the evaluator and gives itself a perfect score?"
🎯 What They Are Testing
The tech colleague uses a sharp counterexample to probe whether you understand the safety design of self-improving systems. Answering "add a prompt telling it not to modify" will immediately lose their respect. What they want to hear is boundary design: what must stay outside the improvement loop, and how each change is validated.
🧭 Answer Framework
  1. Acknowledge the problem is real: This is reward hacking — the most dangerous anti-pattern in self-improvement loops. Optimizing unit tests leads to overfitting test cases; optimizing the judge model teaches gaming; optimizing benchmark scores leads to exploiting benchmark flaws.
  2. Give the boundary principle: Evaluators and permission controls must sit outside the evolution loop, maintained by humans or tamper-proof independent mechanisms. The exam setter and the grader must be independent of the student being tested. The editable surface must have boundaries — editing at the OS system-config level breaks the abstraction boundary.
  3. Give the validation mechanism: Reference the Self-Harness three-stage loop. Weakness Mining clusters failure patterns from failure trajectories; Harness Proposal submits bounded edits; Proposal Validation uses held-in and held-out datasets to verify — only accepting edits with no regression.
  4. Give the capability prerequisite: STOP experiments show that a recursive structure does not guarantee improvement by itself. GPT-4 can continuously improve; weaker models amplify noise and actually regress. Before deploying this system, evaluate whether the base model can support meta-level optimization.
⭐ Bonus PointAdd a risk they may not have thought of: diversity collapse. Evolution loops naturally exploit known high-reward patterns; when all candidates become minor variants of the same solution, innovation stops. You need diversity rewards and archive preservation mechanisms as a safety net. If you can articulate both risks, you win this conversation.
Use these course pages to organize your answer → Let Harness Improve Itself Future Challenges: Seven Gates of Self-Improvement Evolutionary Search: Survival of the Strongest Harness
Q6Interviewer
"Where exactly is recursive self-improvement right now? AI can modify itself now — is it about to get out of control?"
🎯 What They Are Testing
They are checking whether you can evaluate cutting-edge progress without either mystifying or panicking. There are two revealing answers: amplifying "AI is about to awaken," or flatly saying "it's all hype." What they want is industry knowledge grounded in facts, with clear boundaries and judgment.
🧭 Answer Framework
  1. Give the conceptual coordinates first: RSI was envisioned from Good (1965) to Yudkowsky (2008) — a system using its current intelligence to improve the very mechanism that produces intelligence. It remained a theoretical concept for decades; only in the past two years has a real-world path emerged.
  2. Clarify the real-world path: Models do not directly rewrite their own weights. What they improve is the Harness surrounding them: context management, workflows, tool orchestration, evaluation. A better Harness produces a stronger model; a stronger model in turn simplifies the Harness — a positive-feedback flywheel.
  3. Give the true state of progress: STOP proved that an improver can recursively improve itself, and automatically rediscovered classic strategies like genetic algorithms and Beam Search. Self-Harness lets Agents improve their own configurations by mining weaknesses, proposing edits, and validating against regressions. But all of this happens inside bounded, verifiable loops.
  4. Directly answer the out-of-control question: There are still seven gates before "out of control": weak evaluators making feedback signals noisy, poor memory lifecycle management, reward hacking, diversity collapse, and difficulty measuring long-term health. These are fundamental system-design challenges, and the shared solution points toward keeping humans in the loop and providing oversight at the right level of abstraction.
⭐ Bonus PointClose with a reality check on automated research: AI Scientist can produce formally complete papers, but being able to write a paper falls far short of doing science — false citations, implementation drift, and over-optimism recur repeatedly. Use this case to demonstrate that your judgment is grounded in primary sources.
Use these course pages to organize your answer → From Scaffolding to Self-Improving Systems Let Harness Improve Itself Future Challenges: Seven Gates of Self-Improvement
One Last Tip
This chapter has the most cutting-edge knowledge, and it is also the easiest to bluff with. The right way to use these 6 questions is to say your answer out loud — to a colleague, a friend, or a recording. Where you stumble is where you thought you understood but didn't. Click the linked course pages to fill those gaps.