How They Will Test You

LLM Fundamentals · 7 Must-Answer Questions

You've finished Chapter 1 — but there's a gap between understanding and being able to explain it under pressure. These 7 questions come from three real scenarios: try to answer each one out loud before reading the framework.

How to use this page
Each question shows who's asking. They're testing the same knowledge, but they each want to hear something different.
🎙 InterviewerWants to verify whether you truly understand or are just reciting buzzwords
👔 BossWants explanations and commitments
🛠 Tech ColleagueIs probing whether you're worth trusting
Each question has three layers: what they're assessing → answer framework → bonus points. For any part you can't answer, click the linked lesson pages at the bottom to review.
Q1Interviewer
"In your own words, how does ChatGPT and similar LLMs generate responses? Is it actually thinking?"
🎯 What they're assessing
The opening question — it sets the depth of everything that follows. It tests whether you can explain probabilistic next-Token prediction in plain language. Someone who just memorized terms will pile on "Transformer, attention mechanism"; someone who truly understands will walk through the mechanism with a clear example.
🧭 Answer framework
  1. Lead with the essence: An LLM is a massive probabilistic prediction machine. It does one thing at a time: given all existing Tokens, it predicts the probability distribution for the next Token and generates tokens one at a time.
  2. Distinguish training from inference: Training is learning statistical patterns from enormous text data. When you're chatting with it, the parameters are frozen — it's not learning, it's computing.
  3. Address the thinking question directly: It doesn't think in the human sense, yet at sufficient scale it demonstrably exhibits reasoning-like behavior. So neither deify it nor dismiss it as a simple autocomplete.
  4. Ground it with an example: Given "The weather today is really," the model outputs: good 62% / bad 18% / cold 9% — and samples from that distribution. A full response is this action repeated hundreds of times.
⭐ Bonus point Proactively point out that chatting with the model ≠ learning. Many PMs assume the model continuously learns from user conversations. Correcting this misconception shows you've understood it properly.
Organize your answer using these lesson pages → Base Model: Token by Token Training vs. Inference Vocabulary & Training
Q2Interviewer
"It's clearly a chat product — so why is the OpenAI API called chat/completions (completion)? How does multi-turn dialogue actually work?"
🎯 What they're assessing
This is the litmus test for whether you've actually worked with the API. Someone who has only used the ChatGPT web interface can't answer this. Understanding the message list mechanism means you'll be able to handle follow-up questions about context, cost, and Agents.
🧭 Answer framework
  1. Reveal the core: The model is fundamentally a text-completion machine. "Dialogue" means packaging the conversation history in chat-log format and having the model complete the assistant's next turn.
  2. The truth about multi-turn: The model has no memory. Every turn sends the entire message list — system + all previous user/assistant turns + the current question — from scratch.
  3. Add the engineering layer: The model understands conversation format because of Chat Template + SFT instruction tuning — the key step that teaches the base model to "speak."
  4. Elevate one level: Every AI Harness operation (RAG, memory, Agent) is fundamentally about manipulating this message list. Understand it and you've found the entry point to every solution.
⭐ Bonus point Naturally transition to the cost implication: because the full history is resent every turn, multi-turn conversations get progressively more expensive. All downstream cost-optimization solutions are aimed at this.
Organize your answer using these lesson pages → The Mystery of chat/completions Simulating Chat History Chat Template + SFT Why Build Fundamentals
Q3Boss
"Our AI customer service just made up another non-existent refund policy. Explain to me why this happens, and when can we fix it once and for all?"
🎯 What they're assessing
This question isn't testing knowledge — it's testing expectation management. Do you have the nerve to say "we can't fix it completely"? And after saying that, can you immediately offer a reassuring plan with quantifiable commitments? Saying "I'll ask the engineers to look into it" is the worst possible answer.
🧭 Answer framework
  1. Lead with the conclusion, no hedging: Hallucination is an inevitable byproduct of probabilistic prediction. It cannot be fully eliminated, but engineering measures can compress it to a level acceptable for the business.
  2. Then explain the root causes: Two sources: incorrect parametric knowledge (training data was wrong or outdated), and context misinterpretation (the model always picks the most likely continuation, and "most likely" ≠ "most accurate").
  3. Offer a combined solution: RAG to inject real refund policy documents + Prompt constraint "answer only based on the document" + lower Temperature + evaluation harness and human review as backstop.
  4. Give a quantifiable commitment: Define a hallucination rate metric (sampled review), report convergence weekly, and turn "when will it be fixed?" into "when does the metric reach X?"
⭐ Bonus point Add: "Any vendor claiming they can completely eliminate hallucination is overpromising." This helps your boss develop realistic expectations for the entire industry — a core value of an AI PM.
Organize your answer using these lesson pages → LLM Hallucination Demo Summary: Hallucination Types & Root Causes Evaluation + Human Review
Q4Interviewer
"What methods do you know for mitigating hallucination? If resources are limited and you can only deploy one first, which one do you choose and why?"
🎯 What they're assessing
The first half tests breadth of knowledge; the second half is the real test: decision-making ability and cost awareness. There's no standard answer. Someone who jumps straight to an answer fails; someone who first asks "what's the scenario?" passes.
🧭 Answer framework
  1. List all four methods: Prompt constraints (cheapest), RAG (most effective for knowledge hallucination but has cost), lower Temperature (reduces randomness only, doesn't fill knowledge gaps), evaluation + human review (external backstop).
  2. Ask about the scenario first: Is the hallucination mainly fabricating facts, or is the expression unstable? Does the knowledge base change? What's the budget?
  3. Give a conditional answer: Knowledge hallucination (fabricating policies, inventing data) → RAG. Unstable expression → Prompt constraints + low Temperature, nearly zero cost, deploy first.
  4. Add one principle: Regardless of which you choose, establish evaluation first. Without evaluation you can't measure effectiveness — it's wasted effort.
⭐ Bonus point Point out that in real projects all four must be used in combination, staged over time — multiple-choice exists only in interviews. Being able to describe a tempo like "use Prompt and Temperature to stop the bleeding first, then deploy RAG to address the root cause" is rare and impressive.
Organize your answer using these lesson pages → Mitigation 1: Prompt Mitigation 2: RAG Mitigation 3: Temperature Mitigation 4: Evaluation Summary: Decision Framework
Q5Tech Colleague
"The product needs to integrate our latest internal product manual. You're not actually thinking of having us retrain the model, are you?"
🎯 What they're assessing
A half-joking challenge from a tech colleague — actually probing whether you understand the boundary between training and inference. Answer incorrectly (e.g., "Yeah, isn't retraining fine?") and your tech team's trust in you drops immediately. Answer correctly and the collaboration ahead goes much smoother.
🧭 Answer framework
  1. Pick up the joke: No retraining needed. Frozen parameters don't mean knowledge can't get in — the context window is the knowledge entry point.
  2. Give the solution: Use RAG. Chunk the manual, build a vector index, retrieve relevant passages and inject them into context when users ask. Updating the manual only requires rebuilding the index — active in a day, costs orders of magnitude less than retraining.
  3. Clarify the correct use of fine-tuning: Fine-tuning changes behavioral style (tone, format, domain vocabulary) — it's not the right tool for injecting time-sensitive knowledge. Once knowledge enters the weights, every update requires another training run.
  4. Show cost awareness: RAG also has costs: each request uses more tokens, latency goes up. It needs to be paired with caching, routing, and precise chunking for optimization.
⭐ Bonus point Being able to state RAG's trade-offs and optimization strategies (semantic caching, keyword triggers, model routing) shows you've actually done the math. "Use RAG" is something everyone can recite.
Organize your answer using these lesson pages → Training vs. Inference RAG Retrieval Augmentation RAG Trade-offs & Optimization
Q6Interviewer
"What are Temperature and Top-P? How would you set them for your product?"
🎯 What they're assessing
Testing whether you've actually tuned these parameters. The second half — "your product" — is waiting for a scenario-specific answer. Someone who rattles off a fixed value ("just set 0.7") almost certainly has never tuned them in practice.
🧭 Answer framework
  1. Explain the mechanics: Temperature controls how steep the probability distribution is — lower means more deterministic, higher means more spread out. Top-P controls the sampling candidate pool size. Together they determine output randomness.
  2. Give scenario-specific settings: Customer service / factual Q&A / data extraction → lower (0–0.3). Creative copy / brainstorming → higher (above 0.7).
  3. State the boundary: This only mitigates expressive randomness — it doesn't fix knowledge gaps. Even with low Temperature, the model will still confidently make things up.
  4. Provide a validation method: Parameter values should be validated with A/B testing on an evaluation set. Picking one value for the entire product by gut feeling is a bad practice.
⭐ Bonus point Proactively distinguish random hallucination from knowledge hallucination: Temperature only treats the former. Many people can't articulate this boundary.
Organize your answer using these lesson pages → Temperature & Top-P Interactive Demo
Q7Interviewer
"What is a context window? Models are now up to 1M Tokens — is bigger always better?"
🎯 What they're assessing
The first half is a conceptual question; the second half is a trap. Agreeing that "of course bigger is better" falls right into it. This tests cost awareness and whether you know the real limitations of long contexts.
🧭 Answer framework
  1. Clarify the concept: The window = the total number of Tokens the model can see in one pass (input + output). Anything beyond that is truncated — as far as the model is concerned, it never existed.
  2. Expose the trap: A larger window means a larger bill. Tokens are charged by volume. Stuffing everything in makes costs rise linearly.
  3. Add the technical limitation: Long contexts suffer from "lost in the middle": the more information, the more diluted the attention, and recall of content in the middle sections drops significantly.
  4. Give the right approach: A large window is just a capability ceiling. The correct practice is context engineering: retrieve, compress, filter — put only what belongs in the window.
⭐ Bonus point Citing mainstream model window specs (Qwen 1M / Claude 200K / GPT 128K) and noting that window size affects architecture decisions signals that you know the industry well.
Organize your answer using these lesson pages → The Context Window is Key Further Reading (Chapter 2): AI Working Memory
One final recommendation
The correct way to use these 7 questions is to say them out loud — to a colleague, a friend, or a recording. Just reading them doesn't count. Wherever you stumble is where you think you understand but don't — click the linked lesson pages and go back to review.