How They'll Test You

Practice · From Demo to Production · 7 Soul-Searching Questions

You've completed Chapter 3 and now know what separates a Demo from a production product. These 7 questions come from three real scenarios — try answering them out loud first, then check the framework.

How to Use This Page
Each question is labeled with who is asking. They cover the same knowledge, but what they want to hear is different.
🎙 InterviewerWants to verify whether you've actually shipped it or just watched demos
👔 BossWants an explanation, a plan, and a deliverable promise
🛠 Tech ColleagueTesting how big you draw the pie and how much you understand the costs
Each question has three layers: what they're assessing → answer framework → bonus points. For any part you can't answer, click the course pages at the end to review.
Q1Interviewer
"Say your team got the image-generation API working in a single day and the demo looked great. In your view, how far is that from actually launching? What's missing?"
🎯 What They're Assessing
This is the watershed question for judging whether you've actually shipped an AI feature. Someone who has only played with demos will say "just a bit more polish and it's ready"; someone who has shipped knows that getting the API working is only 10%, and the remaining 90% is productization work they can enumerate line by line.
🧭 Answer Framework
  1. Start with the conclusion: Getting the API working is only 10%. In real cases, going from working API to launch took three months. The gaps can be categorized across four dimensions.
  2. Experience layer: Generation progress feedback, one-click retry on failure, multiple results to choose from, history to revisit. In a demo, the user can sit and wait 30 seconds with no feedback; in a product, that's unacceptable.
  3. Quality and engineering layers: Quality relies on Prompt optimization (using an LLM to translate user plain language into descriptions the image model understands) plus character consistency anchoring; engineering relies on multi-model fallback chains, timeout retries, cost caps, and result persistence. The model will definitely fail — the experience after it fails is what makes a product.
  4. Safety layer: Dual content moderation (input and output), copyright risk, and privacy policy for user reference images. A demo can go bare; a production product that goes bare will have incidents.
⭐ Bonus point Mention the universality of this checklist: for any AI feature, the path from Demo to Production always covers experience, quality, engineering, and safety. Hearing that you have a methodology impresses the interviewer far more than just hearing you recite a list.
Use these course pages to structure your answer → Image generation productization checklist Using AI to write Prompts for AI The model will fail — what then?
Q2Boss
"A user reported that the Agent ran for half an hour and never stopped — and gave nothing back in the end. What happened? How do you plan to make sure this never happens again?"
🎯 What They're Assessing
The boss wants both a root cause explanation and a prevention commitment. Saying "the model went haywire" reveals you: it means you can neither articulate the failure patterns nor produce a mechanism to make it stop on its own. Answering "I'll have engineering add a timeout" only gets half credit — that's just the crudest layer.
🧭 Answer Framework
  1. Explain the root cause first: Production Agent deadlocks fall into four typical patterns: same-parameter loops (repeatedly calling the same tool with the same parameters), diminishing returns (50 rounds of nothing but marginal actions), text repetition (rehashing the same content once context gets too long), and cascading tool failures (one tool fails and drags down the whole chain). Start by identifying which pattern this was.
  2. Provide the protection plan: Mistake-proofing is a three-layer net. Hard limits as the safety floor: iteration cap, total timeout, per-tool call count limit — unconditional brakes. Detection and alerts: same-parameter detection, same tool name detection, diminishing returns detection — report any anomalous pattern.
  3. Graceful degradation: When anomalies are detected, first gently correct: inject a note like "you've repeated this 3 times, please try a different approach," temporarily disable the failed tool, force a summary of current progress and return with a partial result. For users, coming back with something partial is far better than empty hands.
  4. Add the experience layer commitment: Even when the Agent is running a long task, users need to see progress — show what it's currently doing and allow manual stops at any time. What users are actually angry about is "I waited half an hour and had no idea what it was doing."
⭐ Bonus point Add: "Users never report that the Agent is in a loop — they just say 'this AI is so slow and stupid' ". Being able to translate a technical failure into the user's perspective makes the boss feel you're the person who can handle this.
Use these course pages to structure your answer → Why Agents get stuck in loops Mistake-proofing: how to make loops stop on their own Streaming UX: don't leave users waiting
Q3Interviewer
"Your AI assistant gets more expensive and dumber the longer the conversation. How would you design context compression? What can be deleted, and what is absolutely off-limits?"
🎯 What They're Assessing
This tests whether you have a compression decision framework and whether you know where the red line is. Answering "just find a model to summarize it" reveals you: you've neither calculated the cost of summarization itself nor realized that deleting the wrong things leads to users discovering "I clearly said that — how did you forget it?"
🧭 Answer Framework
  1. Explain why compression is necessary: Every turn of the conversation re-sends the entire history to the model. The longer the conversation, the higher the cost, the more scattered the attention, the closer to the context window limit — three problems that force you to manage context.
  2. Provide a tiered framework: Deletable: old tool call results, already-processed intermediate steps. Compressible: AI's lengthy replies, search results — compress to a one-sentence summary. Absolutely untouchable: the user's original messages, System Prompt, key preference settings.
  3. Identify the red line: The user's words are sacred. Better to delete 1000 words AI said than to touch 10 words the user said. Compression priority, highest to lowest: tool outputs, AI replies — user messages are never touched.
  4. Clarify the cost of each method: Local compression (truncation, rule-based replacement) is free but coarse; LLM summarization is precise but costs tokens. The correct order is free-first, then paid: use local methods to cut obvious bloat first, then consider LLM summarization for what remains.
⭐ Bonus point Proactively raise the user experience angle: good compression is invisible to users; poor compression makes users feel AI has "amnesia". Treating compression as a UX metric and measuring it transcends the pure cost-saving perspective — very few people offer this angle.
Use these course pages to structure your answer → Longer conversations cost more and get dumber Compression is the art of trade-offs Can the user's words be deleted? Local compression vs. LLM compression
Q4Interviewer
"The product needs to 'make AI remember the user'. How would you design this memory system? Do you store every conversation? What happens when the user changes their mind?"
🎯 What They're Assessing
Three chained questions testing your complete design capability for a memory system: what to store, how to update, how to use. Conflating "memory" with "context" reveals you in the first sentence; answering "store everything and query it later" won't pass the cost and noise test.
🧭 Answer Framework
  1. Distinguish the two systems first: The context window is a whiteboard — fill it and erase it, clear it when the conversation ends. Long-term memory is a notebook — what is written remains the next time it opens. The prerequisite for building a memory feature is accepting that the whiteboard is unreliable.
  2. Design the gatekeeper: Users send dozens to hundreds of messages a day, with "ok", "sure", "haha" making up the majority. The preferences and facts worth remembering are only a few. There needs to be a filtering logic before writing to memory to judge whether information has long-term value.
  3. Handle memory conflicts: A user said last month they like coffee, this month they switched to tea. Four strategies by scenario: explicit replacement → overwrite; complementary information → merge and extend; cannot determine which is correct → mark as conflict pending confirmation; temporary states (lately too tired, sleeping in) → skip and don't store.
  4. Calculate the injection cost: With 1000 stored memories, stuffing them all into the System Prompt every time is simple but expensive and noisy; on-demand retrieval saves money but may miss things. The injection strategy must be chosen based on memory scale and scenario — this is a cost decision.
⭐ Bonus point Point out: "The core capability of a memory system is updating — a memory system that only appends will become a rumor database in two months." Most people only design the write path, not expiration and error correction.
Use these course pages to structure your answer → Context ≠ Memory What's worth remembering and what isn't Memory conflicts: what when users change their mind The cost problem of memory injection
Q5Tech Colleague
"Your PRD says you need multi-Agent collaboration and you drew a pretty cool architecture diagram. Do we really need that many Agents? Can't one Agent handle it?"
🎯 What They're Assessing
The tech colleague is probing whether you've genuinely thought it through or are just chasing trends. Multi-Agent means more coordination cost and more failure points — they will be paying for this complexity. If you can't answer "why a single Agent won't do it," this requirement will most likely be pushed back.
🧭 Answer Framework
  1. Acknowledge the default position first: The default is that one Agent is sufficient. Many so-called multi-Agent scenarios are actually just poorly written Prompts. Before adding a second Agent, answer three questions: can one really not do it? Is the complexity worth it? Is there a simpler solution (like parallel tool calls)?
  2. Name the three genuinely required scenarios: Parallel speedup — searching 5 sources simultaneously is 5× faster than serially. Role separation — Writer writes, Reviewer reviews; role isolation makes the review genuinely effective. Risk isolation — if a sub-Agent fails to parse a PDF it only reports "this file has a problem" and the main task is unaffected.
  3. Map to the scenario: Return to the specific scenario in the PRD and explain which of the three it matches. If it matches, keep it; if it doesn't, cut it on the spot — that looks far better than defending an architecture diagram.
  4. Demonstrate concurrency common sense: Even with multi-Agent, know that "reading" can be parallel and "writing" must be serialized. The one key to judging whether an operation can be concurrent: is it read-only?
⭐ Bonus point Proactively say: "If none of these three scenarios applies, I'll revert to a single Agent." The thing tech colleagues fear most is a PM they have to pay for in slide-deck architecture — naming your own exit builds trust immediately.
Use these course pages to structure your answer → When do you need multiple Agents The cost of concurrency: who can run in parallel Brainstorm: let multiple AIs argue
Q6Interviewer
"MCP is very hot right now. Tell me how it's different from a regular API call. What does it mean for your product?"
🎯 What They're Assessing
This tests how deep your understanding of MCP goes. Answering "a protocol that lets AI call external tools" is indistinguishable from someone who read one explainer article. People who truly understand MCP will explain the bidirectionality: your product can both consume others' capabilities and expose itself as a tool for others.
🧭 Answer Framework
  1. First layer: As a Client, the product consumes external capabilities through MCP — calendar, email, database, browser — connect to one protocol and gain access to an entire ecosystem, eliminating the cost of integrating APIs one by one.
  2. The critical second layer: MCP is bidirectional. The product can also act as a Server, exposing its own capabilities for Cursor, Claude Desktop, or automation scripts to call. One-way integration is just tool calling; bidirectional means your AI can become a tool for others.
  3. State the product implication: When multiple Agents can call each other, an ecosystem forms naturally. This is the key leap from tool to platform, and it's a product positioning decision that PM must make.
  4. Add engineering common sense: Connecting to 10 MCP services at startup all at once? If 3 are down, startup hangs. Registration and connection must be separated — lazy connection (connect when used). Going further, when an Agent discovers it lacks a tool at runtime, it can discover and configure a new MCP connection on its own.
⭐ Bonus point Close with one sentence: "MCP's significance for AI products is analogous to what open platforms meant for mobile internet — figure out first whether you're the integrator or the integrated." Elevating a protocol question to an ecosystem positioning question will make you memorable.
Use these course pages to structure your answer → MCP is more than just "calling tools" Lazy connection: don't connect until needed AI adds its own tools
Q7Boss
"This month's API bill tripled compared to last month, but user growth was only 20%. Where did all the money go? Can we bring it down next month?"
🎯 What They're Assessing
This tests whether you understand the cost structure of Agent products. Someone who thinks cost is proportional to message count can't explain why the bill grows faster than user count. Only someone who can break the bill down to the level of turns and context length can talk about how to reduce it.
🧭 Answer Framework
  1. Correct the unit of measurement first: A user sends one message, but under the hood it may run 10+ loop iterations and dozens of API messages, and every iteration re-sends the full history. Cost is tied to task complexity and grows exponentially with it, so the bill growing faster than user count is normal — out-of-control growth is the problem.
  2. Hunt the typical cost culprits: Scheduled tasks reusing old sessions where context keeps growing — one choice can make monthly bills differ by 10×. Stuck loops burning money on idle iterations. Long conversations without compression, paying for ancient history every turn.
  3. Provide the cost-reduction combination: Switch scheduled tasks to creating a new session each time. Deploy loop safeguards to kill idle iterations. Context compression to cut tokens that shouldn't be resent. Set cost caps by user and time slot, and monitor anomalous calls.
  4. Provide a way to quantify the commitment: Build a monitoring dashboard for per-task average cost. Transform "can we lower it next month" into "per-task cost target and anomalous calls to zero" — report weekly.
⭐ Bonus point Add: "The flip side of cost increases is that over-saving makes the AI dumber, and too much compression degrades experience." Proactively putting the cost-experience trade-off on the table shows you're making product decisions, not just playing accountant.
Use these course pages to structure your answer → The real cost behind one message The cost trap of scheduled tasks Longer conversations cost more and get dumber
One last piece of advice
The common thread across all 7 questions is accounting and risk containment: how to stop a loop, how to compress context, how to control the bill. Being able to explain these fluently proves you've truly walked the path from Demo to production. For anything you can't explain clearly, click the linked course pages to go back and fill in the gaps.