This summer we tried something different; as kids are growing up, we are also realizing we only have 6 more summers with our older one and 8 more summers with our younger one, so we took about a month to visit our family in the Midwest (Cleveland, Michigan, and Chicago) and 10 days to go to Africa to see a whole new world! It was so worth it!

Now that we are back and rested, I am back to learning and eager to share what I've learned. Here are the topics I am diving into in the upcoming week, so that’s what you will learn, too!

1. Automation vs. workflows vs. agents (What are they? How are they different from each other? When to utilize which one? Please join Amy and me as we dive into it with the Seattle Tech Forum Product community on Friday, August 28th.

2. Next, I want to learn more about Agents (I'm still thinking about what that looks like, but I know I'll go beyond just what they are!).

3. This is also on top of my mind, as all of the frontier AI labs keep coming up with the latest and greatest AI layers - the latest one I want to dive into is graph engineering. Look at the timeline below to see when each of the layers came about:

   

Are there any AI topics you are exploring or want to learn about? What are they? Don’t forget to tell about those topics in the comments below.

What you’ll learn (5-minute read):

• What distillation is, in one picture

• Why it works (the “dark knowledge” idea from the researcher who invented it)

• When you’d reach for it as a PM, and when you’d reach for something else

Picture a brilliant professor and an eager student.

The professor is your big model. Brilliant, but expensive to run every single time you ask it something. The student is a much smaller model. It can’t hold everything the professor knows, but it doesn’t need to. It just needs to do one job well.

So the big model teaches the small one. What you get back is a model that’s a fraction of the size and way cheaper to run. And it stays good at the one task you care about.

This isn’t hand-waving, by the way. The classic example is DistilBERT, a distilled version of Google’s BERT model: 40% smaller and 60% faster, while keeping about 97% of the original’s language understanding (Sanh et al., link in Sources). You give up 3 points of capability and get back nearly half the size and more than half the speed. I’d take that trade most days.

That’s distillation in one sentence. The interesting part is how the teaching happens.

Why it works: the teacher shares their judgment, not just the answer

Here’s the piece most explanations skip.

Take a training photo of a cat. Trained the normal way, the student learns from one flat label: cat. Nothing else. Right, but thin.

Distillation trains it on something richer. Instead of that flat label, the student learns from the teacher’s full answer: 90% cat, 8% dog, 2% fox. That spread is the teacher’s judgment, and it’s full of hidden lessons. The 8% dog quietly teaches the student that cats and dogs are kind of similar. The 2% fox says foxes are a little like cats too. A flat “cat” label never teaches any of that.

This idea comes straight from the source. Geoffrey Hinton (yes, that Hinton, the one with the Nobel) introduced distillation in a 2015 paper with Oriol Vinyals and Jeff Dean, and he gave those hidden judgments a name I love: “dark knowledge.” The whole technique is built on the idea of transferring it.

Quick update for the LLM era, because a cat classifier is probably not where you’ll actually meet this word. For today’s language models the teaching often looks even simpler: the small model trains on thousands of the big model’s actual written answers and learns to respond the way the teacher does. Same picture, judgment passing from teacher to student. Only the format changed.

When to reach for it: the PM decision

Distillation is a tool, not a default. Here’s when it’s the right one.

Reach for distillation when the job is narrow and the bill is real. A support classifier handling millions of tickets. An on-device suggestion that has to run on a phone. A high-traffic routing step where a giant model would work but would cost you a fortune. (If you’ve ever watched an inference bill outrun the value attached to it, you know exactly this pain.)

Reach for something else when the situation is different, and this is the part that keeps you from overusing it:

• If you need the big model’s full range across open-ended tasks, keep the big model. Don’t distill away the flexibility you actually need.

• If the problem is that the model needs fresher information, that’s a RAG job (retrieval-augmented generation): you give the model better documents to work from, you don’t shrink it.

• If you just need the model smaller, not retrained, there’s a simpler tool called quantization, which trims a model's numerical precision to shrink it without a full teacher-student process.

Distillation trades a little range for speed and cost. Pick it when the job is narrow and the bill is real.

Why two AI giants fought about it

I promised you a fight.

In early 2025, a Chinese lab called DeepSeek released models that rivaled the best American ones at a fraction of the reported cost. OpenAI’s accusation: DeepSeek got there partly by distilling OpenAI’s own models, using ChatGPT’s outputs as the teacher. OpenAI later told US lawmakers that DeepSeek was making “ongoing efforts to free-ride on the capabilities developed by OpenAI and other U.S. frontier labs”.

Set aside who’s right. What I want you to take from it is simpler: distillation is powerful enough to be a competitive weapon, and that cuts both ways. If your team ever distills from a model accessed via an API, read the provider’s terms first. Most frontier labs explicitly prohibit using their outputs to train competing models. That’s a conversation to have with legal before the training run, not after.

The one honest catch

A distilled model is small on purpose, not small by accident, and the trade is real: it loses some of the teacher’s range on the weird, rare edge cases. Great for the narrow job you trained it for, riskier the further you drift from that job.

Which connects to two things I’ve written before. Distillation is the cleanest example of using the smallest capability that works, the choice that saves you money up front. But the smaller and more specialized a model is, the more brittle it gets when the world shifts under it, which is exactly the drift cost of going small. So distill with your eyes open: cheap and fast today, worth a plan for tomorrow.

That’s distillation. A teacher, a student, and the judgment that passes between them.

If this made distillation click, forward it to one person on your team who could use it. And if there’s an AI concept you want me to explain this way next, quantization, embeddings, RAG, drop it in the comments. That’s how this series gets built.

Next in the series: [to be decided by what you ask for].

Sources

• Hinton, Vinyals & Dean, “Distilling the Knowledge in a Neural Network” (2015): https://arxiv.org/abs/1503.02531

• Hinton’s “Dark Knowledge” talk slides (Google, 2014): https://www.ttic.edu/dl/dark14.pdf

• Sanh et al., DistilBERT — 40% smaller, 60% faster, retains 97% of BERT’s language understanding (paper abstract, arXiv 1910.01108): https://arxiv.org/abs/1910.01108

• Reuters via Bloomberg, OpenAI memo to US lawmakers on DeepSeek distillation (Feb 2026): https://www.investing.com/news/stock-market-news/openai-accuses-deepseek-of-distilling-us-models-to-gain-advantage-bloomberg-news-reports-4504138

AI doesn’t work that way. It’s probabilistic. The same input can yield different outputs. It hallucinates. It drifts. And the most important decision in the whole design isn’t a feature at all. It’s whether AI belongs in the product, and where it has to stop.

No traditional framework asks that. So I built one that does.

I needed something I could run before I started building, something that would stop me from asking “what should I build?” and start me asking “what problem am I actually solving?” I couldn’t find it. So I wrote it, and now it’s the thing I teach my UW students and walk through in talks before anyone touches a tool. Because the fastest way to ship a bad AI product, and the fastest way to fumble an AI design question in an interview, is to assume the answer is always more AI.

It’s called IMPACT. Six layers. I’ll walk all six through one real example: a podcast automation I built and demoed live. It scrapes my AI news sources every morning, has Claude pick and summarize the top stories, turns that into a script, and sends it to ElevenLabs to be turned into audio. Six nodes. I built it in about two hours, my first time ever in n8n. Watch where AI actually shows up.

I — Intent: Name the Burden Before You Reach for AI

This is the layer everyone rushes. Don’t.

When I ask builders for the intent of their AI feature, I usually get something like “make people more productive” or “reduce manual work.” I’ve heard that one enough times to finish the sentence for them. It’s a wish, not a burden.

A real intent is specific enough to act on. It names the actual pain a real person carries. So before you propose anything, answer three questions about the problem itself: What’s the ambiguity AI is handling that rules can’t? What cognitive load is the person drowning in? And what’s the cost of being wrong?

For the podcast, my intent was concrete: the cognitive overload of scanning five-plus AI news sources every single morning. Same tabs, same synthesis, every day. The user wants a spoken digest without having to read it themselves. That’s specific, it’s repetitive, and it’s fully delegatable. Which is exactly why it’s a good automation candidate.

Notice what just happened. I haven’t named a feature yet, and I already know AI has a real job here, because the work is genuine synthesis a tired person repeats daily. If the intent had come back as “make my mornings better,” I’d have nothing to build toward. Vague intent is where most weak answers die in the first thirty seconds.

M — Mental Model: Decide What AI Touches and What Stays Human

This is the layer that makes or breaks AI products, and it’s the one no other framework has.

Once you know AI belongs, you don’t hand it everything. You map the journey and decide, step by step, where AI acts and where a human stays in the loop. Skip this, and six months later, you’ve got an AI doing things it was never supposed to do. Resolving billing disputes on its own. Handing out airfare discounts nobody approved. We’ve all seen those chatbots.

For the podcast, the split is driven by the cost of being wrong, and here that cost is low. A bad summary is mildly annoying, but not catastrophic. So AI can do most of the work, and I made the human review optional, a quick approval before the audio gets generated. That’s the right call when a mistake just means a so-so podcast.

Flip the stakes and the map changes completely. Put AI inside a medical or healthcare flow and you’re not shipping unless it’s 99.99 percent accurate, with a STILL human doing the final check before anything reaches a patient. Same framework, opposite answer. The judgment is in reading the stakes and drawing the line, and that judgment is the thing an interviewer is actually screening for. Anyone can add AI. Knowing where to stop is the senior move.

P — Plumbing: Start Deterministic, Add AI Only Where It Earns It

Now, and only now, the architecture. And this is where the podcast makes the whole point for me.

Six nodes. Pull the RSS feeds, merge them, run a code node to shape the data, send it to Claude to summarize and prioritize the top stories, run another code node to shape the script, then hand it to ElevenLabs for audio and drop it in Drive.

Look at what’s actually happening, tier by tier. The RSS reads, the merge, and the formatting nodes are plain rules, the same deterministic code you’ve always written. The two Claude calls, summarize and write the script, are GenAI. And the ElevenLabs voice is its own kind of AI, a machine-learning model that turns text into speech. So one tiny pipeline already spans three rungs of the capability ladder: rules for the plumbing, GenAI for the language, ML for the voice. And most of it sits on the cheapest rung.

That’s the discipline. You start with the lowest capability that works and climb only when it can’t do the job. Rules first. Then machine learning. Then generative AI. Then agents, then multiple agents, if and only if the problem demands it. The goal is using the right capability at the right moment, not building the fanciest category because it’s exciting. Almost every time, the simpler thing would have worked fine, and the complex thing just ships later, breaks in production, and gets killed six months in.

A — Accuracy & Safety: Design the Failure Before It Ships

Every AI system sometimes produces incorrect outputs. The question isn’t whether it will happen. It’s what the consequences are when it does, and whether you designed for them.

Here’s mine, from the real build. One morning, the script opened with “This is the news from March 24, 2024.” Excuse me. We were in March 2026, and nowhere near the 24th. The model just made up a date and said it with total confidence.

That’s exactly why the human approval node exists in my flow. Before any audio gets generated, the script pauses and shows me what it wrote so I can catch the thing that’s confidently wrong. I also keep the attributions and transcript links verifiable, so I can check that a story is real and not hallucinated. None of that is bolted on at the end. It’s a product decision I made up front, because I already knew the model would be confident even when it shouldn’t be.

C — Cost & Constraints: Don’t Build a $100 Solution for a $5 Problem

Proof of concepts are cheap now. Lovable, Bolt, a dozen low-code tools will get you a working demo fast. The cost question bites later, when you turn that POC into something real and start paying for infrastructure at scale.

For the podcast, I’d run tests for a while and my Claude API bill was about $2.21. Skip one coffee and you’ve covered it. But that’s the POC. The number to watch is what it costs when something works and you scale it up, because that’s where teams burn their runway solving cheap problems with expensive tools. Match the spend to the stakes. A $5 problem doesn’t get a $100 solution.

T — Tracking: The Layer Software Never Had

The AI product lifecycle has one thing the software lifecycle never did: monitoring and maintaining.

You don’t launch and walk away. Once the model is live, you watch what the data tells you. Is it drifting? Does it need retraining, and how often? For the podcast, tracking means the stories stay fresh and I’m not getting duplicates day to day. For a bigger product, it means watching for drift and rebuilding the model as the data moves under you.

And track the promise, not the vanity metric. Engagement is a trap. Time spent could mean the thing is working or could mean it’s broken in a way that makes people keep poking at it. Measure whether you actually solved the burden you named back in Intent.

The One Thing to Take Into Your Next Build, or Your Next Interview

If you remember nothing else, remember the move that makes IMPACT different from every framework before it.

The first real decision in any AI product once you have figured out what to build is whether AI belongs at all, and where it has to stop. My podcast proves it. Six steps across three tiers of the capability ladder, and the real cognitive work lives in only two of them. Most of the rest is the cheapest rung, the deterministic plumbing you’ve always known how to build.

Traditional frameworks skip that question because the software they were built for doesn’t surprise you. AI does. So when you sit down to build, or an interviewer asks you to design an AI product and you open by mapping which tasks go to the model and which stay human, you’ve already shown the thing that matters. Not enthusiasm for AI. Judgment about it.

That’s what I teach before anyone in my class touches a tool. Build the muscle for knowing when to keep AI out, and the rest of the framework does its job.

The podcast was deliberately low stakes. A bad episode is annoying, nothing more. So below, I run the same six layers on a product where being wrong means a hospital visit, an allergy-aware restaurant finder I built for a Demo Day. Same framework, real stakes, and the actual output each layer should produce so you can run it on your own product.

Read more

APIs are metered. APIs are charged for. APIs have rate limits, tiers, overages, the whole thing. That’s been true in software since roughly forever.

So what happens to MCP servers?

Because MCP servers are the new API layer (the way agents reach tools, data, and other systems). Right now, almost all of them are free. Anthropic publishes a directory. Smithery and Glama publish more. You can stand up an MCP server in an afternoon and connect Claude to your CRM, your file system, or your internal docs. Nobody is sending you a bill.

That can’t hold. The shift that’s coming reshapes how SaaS gets priced, how PMs design products, and how the actual unit economics of AI work.

This isn’t a hypothetical future. It’s already starting.

What you’ll learn (10-minute read):

• Why MCP servers can’t stay free forever, and the early monetization patterns are appearing right now

• The structural reason agent traffic breaks human-paced pricing models

• Three pricing models emerging in the MCP economy, with real per-call numbers

• The “dual-identity problem” PMs need to design for (humans plus agents on the same system)

• The governance layer is being built underneath all of this

• What this shift means for AI PM job descriptions in the next 18 months

I Really Think That The Free Period Is Ending

Quick math, because I think it’s the clearest way to see this.

When a human uses Salesforce, they make roughly 50 to 200 actions a day. Open the app, look at a lead, update a field, send an email, close out. Human pace.

When an agent uses Salesforce via MCP, it might make 500 tool calls in 10 seconds while “researching a lead” before doing anything visible. Same infrastructure on the SaaS side. Same database queries, same compute, same egress. Roughly 10x to 100x the load.

The asymmetry is brutal. The bill is going to land somewhere.

I covered the underlying agent architecture in AI Architecture Patterns 101: Workflows, Agents, MCPs, and A2A Systems. What I didn’t cover then (and what I’m trying to figure out now) is what happens to the business model when the architecture changes. Because the architecture has changed. The business model hasn’t caught up.

Apify, Nevermined, and a payments protocol called x402 are all building monetization layers for MCP servers right now. Nevermined offers per-token, per-call, and outcome-based pricing on the same protocol, with sub-cent micropayments starting at $0.001 per transaction. xpay charges in USDC, with tool-specific pricing: search at $0.01, analyze at $0.05, generate at $0.10 per call.

The free period is ending. Probably faster than most people think.

Why the API Analogy Breaks

API pricing was built around three assumptions:

• A developer integrates once and the integration is stable

• Traffic is roughly proportional to human users

• The customer relationship lives at the company level (one company, one contract, one bill)

Agent traffic breaks all three.

The agent integrates dynamically. It might discover your MCP server in a tools list, use it for a single task, and forget about it. Or it might call it ten thousand times in one session. The traffic doesn’t scale with users; it scales with task complexity. And the customer relationship gets tangled because the agent is acting on behalf of someone, but the “someone” might be a person, a team, or another agent.

That last part is the dual-identity problem. I’ll come back to it.

I Can See Three Pricing Models Emerging

Tool invocations (pay per call). The simplest model. Every time an agent calls a tool on your MCP server, you charge. Easy to track, easy to bill. Apify and xpay both use this. It works when the value to the customer scales with how often the agent uses your tools, which is the case in most cases. Where it breaks down: agents are bad at being economical. They retry, they re-query, they loop. A naive per-call model rewards the SaaS vendor for the agent’s inefficiency, and that gets uncomfortable fast.

Re-wrapped human seats. This is the model most enterprise SaaS is shipping right now and they are calling it agent pricing. Microsoft Copilot charges $30 per human user per month for AI features bundled into M365. Salesforce Agentforce add-ons run $125 to $150 per human user per month for "unmetered" agent usage. The seat is still anchored to a person. The agent is just a privilege that the person now has.

This works for procurement (the contract shape is familiar) and it works for the vendor (revenue per human seat goes up). What it doesn’t actually solve: the underlying unit economics. If one human’s agent does the work of three humans, the customer still has only one seat, and the vendor still gets one fee. Re-wrapping the seat doesn’t fix the math. It just postpones the conversation.

True per-agent pricing, where the agent itself is the SKU, isn’t really shipping yet. Intercom Fin makes a point of saying “no seat charges for the AI agent itself” on their pricing page. The fact that they’re calling out the alternative tells you it exists somewhere. It just isn’t mainstream, yet!

Outcome-based pricing. The most disruptive one, and the one I think will reshape the space. Instead of paying per call or per seat, you pay when the agent successfully completes a job. Intercom Fin charges $0.99 per outcome, defined as either a confirmed resolution or a procedure handoff. Zendesk charges $1.50 per automated resolution on committed volume, $2.00 pay-as-you-go.

Deloitte’s TMT Predictions 2026 frames the same direction across the industry: subscription and seat-based licensing are being replaced by hybrid pricing that blends usage- and outcome-based components. The shift is happening in the data, not just on Twitter.

Here’s where outcome-based pricing actually breaks down: defining the outcome is harder than charging for it. When Intercom says “we resolved the ticket” and the customer says “you didn’t,” who decides? There’s no neutral arbiter yet. That gap is going to get filled by an entire infrastructure category that doesn’t exist yet (and might be worth building, but that’s another post).

If I had to pick which model new AI products should default to, here’s where I landed: start with usage-based per-call, design the contract so you can move to outcome-based later. Hybrid is the realistic answer for most teams in 2026.

The Dual-Identity Problem

This is the section I’ve been wanting to write.

When a SaaS product gets used by humans, the design is clear. You build for the human. You optimize for the human. Your pricing reflects how many humans use it.

When the same product gets used by agents on behalf of those humans, everything gets weird.

Picture this. Five employees on your team, each with their own Claude agent connected to your company’s Slack instance. The agents are summarizing channels, drafting messages, and surfacing decisions. At 3am, one employee’s agent runs an automated workflow that pulls all messages from all channels over the last 90 days. It blows through the company’s API rate limit. At 9am, the other four employees can’t use Slack because their team is throttled.

Whose fault is that? Whose seat got “used”? Who pays the overage?

Slack doesn’t have a clean answer. Most SaaS products don’t.

The PM work here is real. You have to ask:

• Does my product treat an agent’s actions as the user’s actions, or as separate?

• If both, how do I attribute cost and usage between them?

• Can I tell, from a single API call, whether a human or an agent triggered it?

• If I can’t, what’s my audit trail actually worth?

These are not edge case questions. They’re going to be table-stakes product requirements in the next 18 months for any SaaS product that opens an MCP server.

Sound familiar? It should. This is the same kind of foundational design choice we made when mobile apps showed up and SaaS products had to decide what “mobile” even meant for their product. Some treated the mobile as a thin wrapper. Some redesigned around mobile-first. The ones who redesigned won.

The Governance Layer Underneath All of This

And then, there’s a second category emerging that’s worth flagging.

Underneath the MCP servers themselves, a layer is forming that handles the messy parts: authentication, rate limiting, cost attribution, audit logging, and agent identity. The standard term for this is “MCP gateway” and it’s being built by Kong, Tyk, MintMCP, Arcade, TrueFoundry, and Microsoft (the AI Gateway inside Foundry).

The CISO (Chief Information Security Officer) version of this story is about security. Palo Alto’s Prisma AIRS framing calls agents “non-human identities” (NHIs) and the risk surface is real. Agents loop. Agents fan out. Agents stay within their rate limits and still create cost spikes that look nothing like human usage.

The PM version of this story is about measurement. Because the gateway is where the data lives. Per-agent, per-tool, per-workflow cost. Per-call latency. Per-outcome success rates. The same data that lets a CISO sleep at night is the data that lets a PM finally answer one question their CFO has been asking for two years: is my AI feature actually saving the company money?

I think this is where the next big category for AI PMs lives, and almost nobody is positioned for it yet. The CISO tools exist. The CFO tools don’t.

So What Does This Mean for AI PMs?

Three things I’d take into next week.

Your product has two personas now. The human user and the agent that integrates with it. One of them is in your wireframes. The other one isn’t. That gap is your roadmap.

Your pricing model assumptions are about to break. If you’re at a SaaS company on a pure per-seat model, the procurement conversation in 2026 is going to get harder. Buyers know the agent-seat math doesn’t work, and they’re starting to negotiate against it. Get ahead of this with hybrid pricing now (a base subscription plus variable usage or outcome components) rather than waiting for renewal to surface the conflict.

There’s a new measurement discipline coming, and it doesn’t have a name yet. Some people call it FinOps for agents. Some call it AI cost attribution. Whatever it ends up being called, the PMs who can answer “what did our agents cost this month, by workflow, by outcome” are going to be in a very different conversation than the PMs who can’t.

I’ll be honest. I don’t have a clean framework for this yet. I’m building one in my head as I write. If you have one, I want to see it.

MCP is Bringing Changes to the AI Economy

The MCP economy is already being built. The free period is ending. The pricing models are being defined right now (mostly by infrastructure companies).

The PMs who notice this in 2026 get to shape what their products look like in 2027. The ones who don’t will inherit the systems built by people who did.

This connects to a frame I keep coming back to: Build AI Products by Capability, Not Category. The MCP economy is a clean test of that idea, because the question isn’t “do I support MCP?” (a category). The question is “what capability does my product offer to agents that humans alone couldn’t have used as effectively” (capability). That’s the product question. The pricing question follows from it.

If this was useful, forward it to one person on your team who needs it. And if there’s something here you’d push back on, or an AI PM question you wish I’d answered, hit reply. I read every reply. The forwards and DMs are the signal that doesn’t show up on any dashboard.

One question I’d love to hear from you: Is your product team treating agent traffic differently from human traffic yet? If not, what’s the blocker? I’m collecting examples for the next post and I’ll credit anyone who wants to be named.

Internal Links Suggested

1. AI Architecture Patterns 101: Workflows, Agents, MCPs, and A2A Systems (foundational context for MCP)

2. Build AI Products by Capability, Not Category (referenced in the closing)

3. To Agent or Not to Agent (good companion piece for readers earlier in their agent journey)

This is Post 2 of 2 in the Drift Series. The playbook. Post 1 was the map. If you haven’t read it, start there — this post assumes the vocabulary from post 1.

In the first post of this series, we built the AI Drift Map. A one-page legend for how AI systems quietly go stale.

The point was simple. When a model “drifts”, something has changed:

• Inputs changed (data/feature drift),

• Rules changed (concept drift),

• Outcomes changed (label drift),

• Plumbing changed (domain / upstream drift), or

• Decisions changed (prediction drift).

That taxonomy doesn’t change when you move from classic ML to cloud-hosted foundation models like GPT, Gemini, or Claude. What changes is who controls what, and which levers you can pull when drift shows up.

This post is about that control plane.

Foundation models vs your own models, in drift terms

When I say “building on foundation models,” I mean three common patterns:

• Calling a cloud-hosted foundation model API directly (OpenAI, Anthropic, Google).

• Using a managed foundation model on a cloud platform (Bedrock, Vertex, Azure OpenAI, watsonx).

• Fine-tuning a foundation model, but still running it on the provider’s infrastructure.

When I say “your own models,” I mean:

• Classic ML models, you train and deploy.

• LLMs or foundation models you self-host and fully control (weights and infrastructure both).

Same physics. Two different architectures. And the architecture decides which knobs you can turn.

Same drifts, different ownership

The high-level view.

Here’s the table I’d hand a PM walking into their first AI architecture review.

The takeaway, in one line:

Two drift stories

Let’s make this concrete.

Story 1: Drift on your own model (a churn model)

You ship a churn model for a B2B SaaS product.

• Inputs changed: you expanded from U.S. enterprise customers to SMBs in new regions.

• Outcomes changed: churn rate jumps from 4% to 10% after pricing changes.

• Decisions changed: the model still labels accounts as “low risk” that are now clearly at risk.

Your runbook:

• Update labels and calibration targets.

• Retrain on newer data with the new mix.

• Add features that capture the new dimensions (region, segment).

• Update thresholds and monitoring.

Everything lives inside your stack. You can change data, model, or both.

Story 2: Drift on a foundation model (a support copilot)

You ship a support copilot built on a cloud-hosted foundation model with RAG.

• Inputs changed: support tickets shift from “how-to” questions to “complex billing” and “policy edge cases”.

• Plumbing changed: your RAG index hasn’t been refreshed in months. New policy changes are missing.

• Rules changed: the provider updates the hosted model sometimes without a version bump on your end, and behavior shifts, becoming more cautious or simply different in the cases you care about.

• Decisions changed: the copilot becomes more evasive and less helpful on exactly the hard cases you care about.

Your runbook is different:

• Prompt drift: tighten system prompts and policies.

• RAG drift: refresh indexes, update retrieval filters, seed your evals with new documents.

• Provider drift: run golden prompts and evals on a schedule, catch behavior changes early, switch model versions or vendors if needed.

You never touch the weights. You have plenty of room to steer behavior.

The Drift Readiness Checklist

Five steps. Doable on a normal sprint.

Step 1: Inventory your AI surfaces. List 2-3 critical AI use cases (fraud, churn, recommendations, copilots). Label each as “own model,” “foundation model,” or “hybrid.” If you can’t get to three, one is fine. Start with the highest-stakes one.

Step 2: Minimal monitoring. Log inputs, outputs, and at least one business KPI. Add one simple drift signal for inputs, outputs, and outcomes. None of this needs to be elaborate. A nightly job and a Slack alert are enough.

Step 3: Golden tests. For foundation models, create 10-50 golden prompts with expected behavior. Run them on every change, plus weekly. For your own models, maintain a basic eval set with labels. (I wrote a walkthrough on building your first eval that lays this out step by step.)

Step 4: One-page runbook. For each drift box (inputs, rules, outcomes, plumbing, decisions), define two things: who investigates first, and what the first lever is. Prompts/RAG vs data/model/pipeline.

Step 5: Governance. Make the AI Drift Map and the Drift Readiness Checklist part of your design reviews and postmortems. The map is only useful if your team actually uses it.

I turned this into a one-page Drift Readiness Checklist you can paste into your workspace and use in design reviews. Grab it here.

What you’ve got after two posts

You now have:

• A map (Post 1): the vocabulary and visuals for all the ways AI systems drift.

• A guide (Post 2): how drift behaves on your own models vs cloud-hosted foundation models, and what to do about each.

Use the map when you’re explaining drift to stakeholders, students, or new teammates. Use the runbook when you’re planning your AI architecture or reviewing a production incident. Use the checklist when you’re not sure where to start.

I wrote about evals and synthetic data earlier this year, and both of those land harder once you have the drift map in your head. If you haven’t read them, they’re the next stops in this same conversation.

Related posts

• AI Drift Map: How Your AI Systems Quietly Go Stale (Post 1)

• Synthetic Data 101

• How to Build Your First AI Eval

• Revising My 2024 Guardrails Post

If this was useful, forward it to one person on your team who needs it. And if there’s something here you’d push back on, or an AI PM question you wish I’d answered, hit reply. I read every reply. The forwards and DMs are the signal that doesn’t show up on any dashboard.

What’s the one AI PM resource you’d hand to a non-engineer PM trying to skill up? Drop it below, I’ll compile.

This is Post 1 of 2 in the Drift Series. The map. Post 2 is the “what to do when drift hits you”, depending on whether you own the model or build on a cloud-hosted foundation model like GPT, Gemini, or Claude.

Drift is where AI products quietly die after launch. This post is the PM’s map: the five kinds of drift, how each one shows up in production, and what to ask about each in a design review.

The AI Drift Map

Think of the AI Drift Map as a one-page legend for all the ways your model can go stale.

At the top, it asks a simple question: what changed?

• Inputs changed → Data/feature drift

• Rules changed → Concept drift

• Outcomes changed → Label drift

• Plumbing changed → Domain / upstream drift

• Decisions changed → Prediction drift (what you see first)

This vocabulary is intentionally PM-friendly, but it maps directly to the formal definitions used in ML monitoring tools and research.

Let’s walk through each box with one example you can reuse in your own projects.

Inputs changed: data drift

Data drift, also called feature drift, occurs when the inputs your model sees in production no longer resemble the data it was trained on.

Think of a coffee shop that opened to serve weekday-morning office workers and slowly became a weekend brunch spot for students. Same menu. Same espresso machines. Same staff. Different crowds walking in, ordering different drinks at different times, sitting longer. You’re still “doing coffee.” But the crowd you optimized for isn’t who walks in anymore.

Industry example: You trained a churn model on U.S. enterprise customers. A year later, your business expanded into SMBs across new geographies. Company size, contract length, and spend - the feature distributions all shifted. The model’s still internally consistent. But it’s making predictions about a population it never really saw.

What changed: the inputs. What’s still intact: the model and its rules.

The diagnosis question to ask in a review:

“Did our user or data mix change meaningfully since we trained this?”

Rules changed: concept drift

Read more

Their proposed fix: take whatever real data the businesses gave them, generate synthetic data on top to fill the gaps, and train the model on the combined set. Use synthetic data as a bridge from “not enough” to “enough to start.”

I told them that was exactly the right instinct. I also told them they were stepping into a bigger conversation than they realized.

Here’s what I dug into so they (and you) could think about it clearly, and actually act on it.

What you’ll learn (12-minute read):

• What synthetic data actually is (and what it isn’t)

• The three situations where it becomes your best option

• How it gets generated, from rules to foundation models

• How teams actually generate synthetic data today (with a few tools to look into)

• Where it’s showing up in real products

• The risks you need to design around, and how to validate against them

• A practical framework for deciding if your product needs it

What Synthetic Data Actually Is

Synthetic data is artificially generated data that mimics the statistical properties of real data without intentionally including real records. (More on the “intentionally” qualifier later. Leakage is a real risk.)

That sentence is accurate. It’s also not very helpful.

Here’s what made it click for me: synthetic data is a stunt double. It looks enough like the real actor to film the dangerous scenes, but nobody’s actual body is at risk. The audience can’t tell the difference in the final cut. But if the stunt double doesn’t move like the real actor, the scene falls apart.

The same trade-off applies. Synthetic data is useful exactly to the degree that it faithfully represents the patterns in your real data. When it does, it’s genuinely powerful. When it doesn’t, you’ve trained your model on fiction.

Three Situations Where Synthetic Data Becomes Your Best Option

Not every team needs synthetic data. But these three situations keep showing up, and when you’re in one of them, real data alone won’t get you there.

1. You Can’t Use Real Data

Healthcare. Finance. Education. Any domain where the data you need is personally identifiable, legally protected, or ethically sensitive. HIPAA, GDPR, FERPA. The acronyms change, but the constraint is the same: you can’t just hand real patient records to your ML pipeline.

Healthcare AI is the textbook case. The Synthea project, an open-source generator of synthetic patient records, exists for exactly this reason. Real records can’t be shared across institutions; synthetic records can. Teams that need to test model behavior on edge cases (rare diseases, complex polypharmacy, demographic representation) generate test populations rather than wait for permission to access real patients’ data. Synthea is widely used in research and benchmarking.

The pattern repeats outside healthcare. Anywhere data is regulated, synthetic data is how teams prototype and test.

2. You Don’t Have Enough Real Data

This is the rare-event problem. Your model needs thousands of examples of something rare. Reality has given you a hundred. This is called class imbalance, and it’s one of the most common problems in applied ML.

The numbers are brutal. In fraud detection, legitimate transactions might outnumber fraudulent ones 10,000 to 1. In manufacturing defect detection, you might have millions of “good” images and fifty “defect” images. In autonomous driving, the scenarios that matter most (near-misses, edge cases, unusual weather conditions) are by definition rare.

You can’t train a model to catch something it’s barely seen. Synthetic data fills the gap by generating thousands of realistic examples of the rare class. In practice, teams often combine techniques such as SMOTE (Synthetic Minority Oversampling Technique) style oversampling or GAN-based tabular synthesis (e.g., CTGAN) to expand the representation of rare classes. More on these in the methods section below.

3. Your Product Doesn’t Exist Yet

This is the cold-start problem. You’re building a product for a market that doesn’t have existing data, because the product itself is new.

A robotics team building a warehouse navigation system for a new facility. An autonomous vehicle team preparing for road conditions in a country they haven’t launched in yet. An AI assistant being designed for a workflow that no one has done before.

And, at a smaller scale, the UW students from the opening of this post: a B2B product team that can pilot with a handful of customers but needs thousands of records to train. Same problem, different size. The product doesn’t exist yet, so the data doesn’t either.

You can’t collect user interaction data for a product that hasn’t shipped. But you need that data to build the product that ships. Synthetic data breaks the chicken-and-egg problem.

I touched on this in my World Models 101 post. The sim-to-real pattern: train heavily in simulation, then fine-tune on smaller real-world datasets once you deploy. Synthetic data is what makes that pattern possible.

That’s the “what” and the “when”: what synthetic data is, and the three situations where you actually need it.

The rest of this post is “the how” and “the decide”. The operational half you run on Monday morning:

• How synthetic data actually gets made: the four generation methods, each with a PM checklist for when to use it

• How teams generate it today: three concrete workflows, including a no-code LLM workflow you can run yourself, plus the vendor tools worth a look

• Where it’s showing up in real products: autonomous vehicles, healthcare, finance, evals, and LLM training

• The four risks, each with a validation check: how to catch distribution mismatch, artifacts, privacy leakage, and bias amplification before they ship

• A decision framework: whether your product needs synthetic data at all, and which method fits your risk tolerance

• Three interview questions: how to handle synthetic data when it comes up in an AI PM interview

Read more

Quick note before this one: this is different from what I usually write here. I’ve been quiet for about a week and a half. I’ve been heads-down the last month building something rather than writing about it. I tested it on myself first. This week I’m shipping it. Here’s how it went.

Let me tell you about this context portfolio I built, and how I got started. How many of us business owners and side hustlers have answered the same questions over and over in different tools? This context portfolio is going to be your one-stop shop, believe me - even if you don’t believe me - keep reading :)

Before I started writing this blog post, I spent 75 minutes being interviewed about my own side hustle by a structured interview I built myself.

The system prompt, templates, multi-file capture, and judgment-layer challenges. All mine. Claude is the engine that executes them.

The interview asked me things I thought I had ready answers for. Then it pushed back when those answers were too generic.

Mid-interview, a sentence came out that I’d never assembled before. It compressed three things at once. Who I am, who I’m writing for, and why now. The interview flagged it as load-bearing, and I had to thread it through three other files before continuing. That’s not something you do alone. You skip past sentences like that when you’re writing for yourself.

Then the interview challenged my thesis. Not gently. Three counter-arguments came back, all real, all things I’d been quietly hoping nobody would push on. I kept the core thesis intact but rewrote the framing to focus on the strongest of the three. The new version is harder to knock over. That update would have taken me months on my own.

Then the coherence check caught a gap between my long-term goals and my current bets. Stated it plainly: partial, with an honest gap. The kind of thing you’d never write into your own document because it makes you uncomfortable.

At the end, twelve markdown files sat in a folder on my laptop. My strategic context. Confidence-tagged. Portable. Ready to drop into any AI tool I use after this.

That’s what I built. I open-sourced it today.

github.com/aipmguru/founder-context-portfolio

A non-tech founder who ran the portfolio with me this week put the value prop in his own words:

“An assistant that just understands everything about your company. And any decision that you can bounce it off of this system. And it’ll make sure that with whatever you choose, it’ll align with your ultimate end goal.”

You never have to re-explain your business to another AI tool again. The twelve files are the how. The persistent memory is the why.

The problem this solves

You’ve explained your business fifty times this year. To investors. To advisors. To a friend at dinner. To ChatGPT. To Claude. To Cursor. To your cofounder.

Every time you open a new AI tool, you start over.

You write a prompt that tries to compress everything: “I run a Series A SaaS for healthcare ops, my customer is hospital revenue cycle teams, my pricing is per-seat with a usage component, here’s where I’m stuck...”

Then the next tool, the same paragraph in a slightly different shape, with the small things you forgot to mention last time. A week later you re-explain to a new advisor and forget the thing you told ChatGPT yesterday.

Your strategic context lives in your head and shows up in fifty different ChatGPT threads, none of which talk to each other. The answers reflect that. Every AI tool you use is having a generic conversation with you because you keep feeding it generic prompts. You get generic answers. You scroll, paste, copy, rewrite, and start over the next day.

This is the system that ends that.

What’s in the repo

Twelve markdown files in one MIT-licensed repository. Free.

The files capture:

1. Identity — who you are, what you do in plain English

2. Founder-Market Fit — the earned secret you have that competitors don’t

3. Market and Customers — your beachhead, customer language, what they’re using today instead of you

4. Thesis and Bets — what you believe has to be true, what you’re betting on, what would kill the bet

5. Positioning — competitive alternatives, category, unique value, point of view

6. Narrative and Story — origin, public version, private truth, voice

7. Business Model — value, pricing, unit economics, what breaks the model

8. Constraints — runway, time, team, stack, ethical lines, refused-to-do list

9. Goals — week, month, quarter, year, long-term, and what you’re explicitly not chasing

10. Stakeholder Pressure — who’s pulling on you and where their expectations conflict with your thesis

11. Decision Log — recent decisions, options considered, reasoning, what would change your mind

12. Competitive Landscape — direct competitors, emerging threats, your moat, response to a $50M-funded entrant

You don’t fill these in by hand. You run a guided interview that lives in interview-protocol/agent-system-prompt.md. You run it in Claude Code, Claude.ai, ChatGPT, Cursor, Gemini. Anywhere that reads markdown files.

The interview pushes back on weak answers. It tags every claim as Evidence / Faith / Untested. It runs devil’s advocate on your thesis. It captures the moments your private truth diverges from your public narrative. When you give it a generic answer, it asks again.

It’s not a fill-in-the-blank template. It’s a structured interrogation (yes, I know I said interrogation, and you’ll get that when you’re going through the interview - believe me) that ends with strategic clarity you actually wrote down.

Two audiences, one context portfolio

Founders making strategic calls without a PM. Solo founders. Cofounder duos. Pre-seed and seed-stage operators who are scoping AI features, pricing decisions, customer theses, and positioning. All without a senior PM in the room. The portfolio gives you the strategic thinking layer that lives in markdown. The interview does what a good PM would do if you hired one: ask the questions you’ve been avoiding.

AI PMs and product operators building a body of work. If you’re an AI PM with frameworks, posts, courses, interview prep, and conference talks, your strategic context (thesis, positioning, decision log) is also worth structuring. The same twelve files capture your professional context the way they capture a business’s. You take them with you across roles, contracts, side projects.

Same twelve files, two different audiences. That’s the point of building it markdown-first.

Why does this matter more than another founder framework?

Three reasons.

1. AI tools amplify what you give them. Generic prompts produce generic answers that are already in their training data. Structured, confidence-tagged, contradiction-checked context produces advice calibrated to your situation. The quality of your AI conversations is bottlenecked by the strategic clarity you’ve actually done the work of writing down. Most founders haven’t.

2. Strategic context is the most valuable thing in your head and the worst-organized. You make thirty product decisions a month. By month eighteen, you can’t remember why you decided what you decided. You can’t remember which advisor pushed for which thing. You can’t remember why you killed the feature you killed.

The decision-log file alone is worth the time you spend on it. The thesis-and-bets file forces you to write down what would make you kill the bet. Most founders don’t have that on paper anywhere. They have a deck. And the deck is the polished version of strategic thinking that already happened, sanded smooth for an audience. The portfolio is the actual thinking, with the rough edges still in it. That’s what makes it useful for the next decision instead of the last fundraise.

3. Markdown beats every other format. Your strategic context shouldn’t live inside a vendor’s UI. It should live in your laptop and your GitHub. No vendor lock-in. You own the artifact.

How to use it

Five steps. The whole thing takes 60 to 90 minutes if you have most of the strategic context already in your head. Longer if the interview surfaces gaps you need to think through.

Step 1: Click “Use this template” on the GitHub repo.

Go to github.com/aipmguru/founder-context-portfolio. Click the green “Use this template” button at the top right. Choose “Create a new repository.” This makes your own private copy. Your strategic context never touches my repo.

Step 2: Create your own private copy.

Name it whatever feels right. I named mine aipmguru-context. Make it private. Click create. You now have your own repo with the 12 empty files waiting.

Step 3 — Get the repo onto your computer

Two options — pick whichever you’re more comfortable with.

Option A — Download ZIP (easiest, no terminal required)

1. Go to your new repo (github.com/[your-username]/my-context-portfolio)

2. Click the green Code button (top right of the file list)

3. Click Download ZIP

4. Unzip the downloaded file into a folder you can find easily — Documents, Desktop, wherever you keep important files

Option B — Clone with Git (terminal users)

git clone https://github.com/[your-username]/my-context-portfolio.git
cd my-context-portfolio

Then: open the folder in Claude Code

1. Open Claude Code

2. Click Open Folder (or File → Open Folder)

3. Navigate to where you saved or unzipped your repo

4. Select the folder

Note

You should now see the folder structure on the left side of Claude Code: templates/, interview-protocol/, plus README.md and other docs.

Step 4 — Start the interview

In the Claude Code chat window, type this and press Enter:

start interview

Claude Code reads the file at interview-protocol/agent-system-prompt.md and begins the structured interview.

Four tips for answering well:

• Be specific. “Mid-market SaaS” is not specific. “Series A SaaS companies with 50-200 employees in the last 18 months” is.

• Don’t polish. This is for you and your AI tools. Nobody else reads it. Polished answers hide assumptions.

• It’s okay to say “I don’t know.” That tags as Untested and gives you a list of things to validate.

• Use the confidence tags. Evidence / Faith / Untested. The discomfort of marking something Faith or Untested is exactly the point.

What I actually went through

I ran this interview on AIPMGURU this week. A few things happened that wouldn’t have happened in a self-guided journaling session.

The interview pushed back when I hand-waved. Early on I gave a generic dinner-party description of the business. The pushback came right back: “That’s the about-page version. It could be ten different newsletters. Drop the marketing layer. Who’s actually reading you, and what do they get from it that they can’t get from the fifty other AI PM newsletters?” That single push got me to a beachhead persona, three reader segments ranked by need, and a specific data point about which posts grew the list versus which didn’t. I would have written a smoother version in a self-guided document. The interview wouldn’t let me.

Devil’s advocate updated my thesis. I’d been carrying around a thesis framing with a single urgency anchor. The interview surfaced three counter-arguments. I absorbed one of them and rewrote the thesis around a more durable framing. The new thesis travels across geographies and survives the original anchor. That’s the kind of revision that only happens when something you can’t dismiss is asking the question.

Coherence check caught a gap I’d been ignoring. My long-term goal doesn’t yet stack with my current bets. The interview stated it plainly. Not as a criticism, as an observation. Once I read it written down, I couldn’t un-see it. That’s now the right strategic question for me to work on.

Confidence tagging surfaced what I don’t actually know. Every claim gets tagged Evidence, Faith, or Untested. I caught myself wanting to tag a few claims “Evidence” that were really “Faith.” Once you have to label it, you can’t dodge it.

Here’s a thing I didn’t expect when I built this: the interview doesn’t only catch things you don’t know. It catches things you know perfectly well but haven’t said out loud yet. The avoidance is what surfaces, not the ignorance. That was a different reaction than I’d planned for.

The discomfort is the value.

A founder who ran the portfolio sent me this:

“This AI system is hardcore, it really made me do some deep thinking about my current business, how to approach prospects, and how to pivot or approach all these AI changes happening to startups.” — Jennifer, GoWavey

Sharpened differentiation. Caught contradictions. Strategic clarity you can hand to any AI tool. That’s the experience.

What sits on top of the files

The repo is free. MIT-licensed. Fork it, edit it, use it however you want.

What sits on top of the files is the Strategic PM OS. A workshop where I teach a set of structured AI Skills that read your portfolio and walk you through every major strategic decision. Customer thesis. Problem validation. Bet selection. Coherence check. And the rest of the operating system you’d build if you had a product team. Each skill produces a decision memo you can defend to your board, your cofounder, or yourself.

Building the portfolio is step one. Running an operating system that uses it to make ongoing strategic decisions is step two.

Next cohort: Saturday May 30 and Saturday June 6, 2026. Four hours each. Virtual. Founding cohort rate: $597. Fifteen seats only.

A founding-cohort student told me:

“Those courses give you knowledge. This gives you a working product built around your specific business.”

That’s the bar I’m holding the workshop to.

Reserve your seat → pmos.founderwell.com

What to do now

Pick whichever fits where you are:

1. Star the repo. Helps me know what’s landing and who’s reading.

2. Use the template. Click “Use this template” on the GitHub page to create your own private copy. Cost: zero dollars. Your business context never leaves your machine.

3. Run the interview. Sixty to ninety minutes of structured pushback. Twelve filled files at the end.

4. Sign up for the PM OS workshop. May 30 and June 6. Fifteen seats. The judgment layer that sits on top of the files.

Get the repo → github.com/aipmguru/founder-context-portfolio

If you run the interview, reply to this post with what surfaced. The load-bearing phrases other founders find are interesting to me, and I’ll write about the patterns I see across portfolios as more come in.

A short note on lineage

This is built on top of Nathaniel Whittemore’s Personal Context Portfolio. I adapted it for founders and strategic operators. The differences: founder-strategic content (not personal-general), multi-file capture during the interview (one answer feeds three files at once), judgment-layer challenges woven in (not just neutral capture), and integration with the Strategic PM OS.

Credit where it’s due. The structure is portable. The founders who run it are the ones who make it real.

The discomfort is the value. Build the portfolio anyway.

I wrote about guardrails on this Substack in February 2024. I organized that post by category. Technical guardrails. Data-driven approaches. Process and human oversight. Legal and ethical. Four buckets of what kind of guardrail you’re building. That cut held up when AI products mostly generated text.

The headlines from the past eighteen months have a pattern. An agent deletes production data. A coding agent silently burns through hundreds of dollars in API calls overnight. An agent retrieves customer records that nobody told it were off-limits.

Every one of those incidents traces back to the same gap. Not “we forgot a category.” We didn’t know when, in the system’s journey to fire, each guardrail would be fired.

Two years ago, the right question was what kind of guardrail does this product need? Today, the right question is when in the system’s journey does each guardrail fire? The 2024 categories are still valid however, they need to be re-cut by stage.

What Guardrails Meant in 2024

Two years ago, the AI PM world was mostly shipping chatbots. LLMs that generated text. Maybe a RAG pipeline. Maybe a function call or two if you were feeling ambitious. The product surface was LLMs generating words.

So the way I organized guardrails in 2024 made sense for that surface. I cut by category. Technical guardrails covered the model’s output (toxicity filters, length enforcement, fact-checking). Data-driven approaches covered training and content quality. Process safeguards covered human oversight. Legal and ethical covered compliance and impact. Four buckets, each addressing a different kind of risk.

When the surface is words, organizing by category works. Most of your risk is concentrated at one moment, the model produces a response, and your category buckets cover the kinds of things that can go wrong with it. The journey only has one moment that matters, so it doesn’t matter much that the categories don’t tell you when anything fires.

But the surface area of what an AI product does has expanded since then. The journey isn’t one moment anymore.

What Changed in Two Years

Your product surface stopped being words, now in 2026 - it is all about actions. It books calendar events. It writes code to your repo. It exports data. It executes transactions. It touches external systems your customers depend on.

That shift is the whole post I wrote about last week: AI Agents vs. Agentic AI. Five dimensions of autonomy, dialed up from “chatbot with search” to “autonomous system that runs for hours.” When you turn those dials up, your failure modes change completely.

Here’s the part that matters for guardrails: when your product only generates text, a guardrail failure is a bad response. Embarrassing, maybe reputationally damaging, but bounded. You catch it and move on.

When your product takes actions, a guardrail failure is an incident.

An agent that exports a customer list isn’t a bad response. It’s a data breach. An agent that modifies a production database isn’t a quality issue. It’s an operational event. An agent that runs up $40,000 in API costs overnight because nobody capped the spend isn’t a miss. It’s a finance conversation with your CFO.

A 2024 stack was built for the bad-response failure mode. The 2026 failure modes need their own stack.

The Four-Layer Guardrail Stack

The mental model I landed on after two years of watching this play out, in class, in interviews, in the agentic features I’ve built: guardrails in 2026 are a stack of four distinct layers, each firing at a different moment in the system’s journey.

Let me walk through each layer.

Stage 1: Before the Model (Input Guardrails)

These fire before the model sees anything.

• Prompt injection detection. Someone is trying to hijack your system prompt. Catch it before the model reads it.

• PII masking. The user pasted a credit card number, an SSN, a customer ID. Mask it before it hits the model and before it gets logged anywhere.

• Off-topic rejection. A customer support bot is being asked to write code. A medical Q&A tool is being asked about stock picks. Recognize scope violations and reject at the door.

• Policy and scope checks. Does this user have permission to ask this question at all? Is the request in the approved use case?

This is the layer most teams have. It’s the closest thing to the 2024 guardrails. A well-known set of filters, solid tooling and clear product decisions.

Most AI PMs can list these without thinking.

Stage 2: During (Context Guardrails)

These fire while the model retrieves and reasons.

This is the journey moment the 2024 framework didn’t cover well. The data-driven category in that post addressed training data quality, not runtime data access. In 2024, most products weren’t doing retrieval at scale, and most PMs treated retrieval as an engineering concern rather than a product one. That’s the gap.

Context guardrails control what the model can see. And that’s a product decision, not an engineering one.

• Data access controls. If the agent is doing RAG, it should only retrieve documents that the user is authorized to see. A support agent looking up ticket history shouldn’t pull up tickets from a different account.

• Row-level security on retrieval. Vector databases don’t enforce this for you. Your retrieval layer has to filter based on user identity.

• Memory scoping. If your agent has persistent memory, whose memory is it? What does it retain? When does it forget?

• Permitted data scoping. Even authorized data has a scope. Personal data from three years ago might be authorized technically, but off-limits by policy.

Why most teams miss this layer: it looks like plumbing. It feels like infrastructure. It gets pushed to engineering as a data security concern and stays there.

But if an agent retrieves data it shouldn’t have seen, the incident is a product incident. Your name is on that PRD. Context is a guardrail.

Stage 3: After the Model (Output Guardrails)

These fire before the response goes back to the user.

This is the 2024 layer. The one everyone has. The one most teams cover thoroughly.

• Hallucination detection. Cross-check outputs against retrieved sources. Flag claims that don’t trace back to a verified context.

• Toxicity and safety filters. Block harmful content before it reaches the user.

• Compliance checks. Financial outputs need disclaimers. Medical outputs need “consult a doctor” language. Legal outputs need a jurisdiction context.

• Format validation. If the output is supposed to be JSON, make sure it parses. If it’s supposed to be under 200 words, enforce that.

• Brand voice checks. Does this response sound like your brand, or does it sound like generic GPT?

These are critical. They’re necessary. They’re also not sufficient.

A system with perfect output guardrails and nothing else can still export the wrong customer list. Can still charge the wrong card. Can still send the wrong email. Because by the time you’re filtering the output, the agent has already decided to take an action.

That’s what Stage 4 is for.

Stage 4: Before Actions (Behavioral Guardrails)

These fire before the agent does anything in the real world.

This is the moment of the journey that didn’t exist in most 2024 products. The process category in that 2024 framework covered human oversight in general, but “before the agent takes an action” wasn’t a moment that needed its own layer because most products didn’t have that moment. Now they do, and most teams skip Stage 4 entirely.

• Tool permissions. What tools does the agent have access to? What’s out of scope? An agent that can read from your database probably shouldn’t be able to drop tables.

• Confidence thresholds. Only execute if the agent is above X% confident. Below that, flag for review.

• Human-in-the-loop triggers. High-stakes actions always get a human. Financial transactions over $Y. Customer-facing emails. Permanent deletes.

• Cost caps. Maximum spend per task, per user, per hour. Circuit breakers when limits are hit.

• Rate limits. Not just on API calls. On the number of actions per unit time.

• Rollback capability. If something goes wrong, can you undo it? If not, you need stronger gates before the action.

This layer is where the real 2026 incidents happen. Every story you’ve heard about an AI agent that “went off the rails” traces back to Stage 4, missing or incomplete.

The pattern is almost always the same. The team had input filters. Had output filters. Built the agent. Shipped. Nobody wrote down what the agent was permitted to actually do. The agent did a thing nobody expected. Incident.

Why Most Teams Miss Stages 2 and 4

Teams with AI products in 2026 usually have Stages 1 and 3. Input filters and output filters. The 2024 version of guardrails.

They usually miss Stages 2 and 4.

Why? Because Stages 1 and 3 feel like the model’s safety, Stages 2 and 4 feel like infrastructure.

Context guardrails look like data security. Behavioral guardrails look like DevOps. Both get scoped to engineering, get pushed to the backlog, and never land in the PRD because they don’t feel like product decisions.

They are product decisions. They might be the most important product decisions you make on an agentic feature.

What the agent can see (Stage 2) and what the agent can do (Stage 4) aren’t technical constraints. They’re the literal scope of the product. If you wouldn’t let an engineer decide what features ship, you shouldn’t let them decide what an agent can touch and what it can’t.

Here’s the gut check I use now. If someone asked you, “What’s this agent allowed to do and under what conditions?” and you can’t answer in three sentences, you don’t have Stage 4. And if you can’t answer “what data is this agent authorized to see?” the same way, you don’t have Stage 2.

Two questions - that’s it!

The PRD Pattern for 2026 Guardrails

So what does this actually look like when you’re writing the spec?

I wrote a bigger post last month on The Builder’s PRD. The five table-stakes PM questions and three AI-specific ones. Guardrails is the section most AI PRDs are missing entirely, and when it is there, it’s usually a short paragraph about toxicity filters.

That’s not enough anymore. The guardrails section needs to answer four questions explicitly. One per layer.

Input (Stage 1): What inputs does the system accept, and what gets rejected at the door?

Context (Stage 2): What data is this system authorized to see, and who enforces that boundary?

Output (Stage 3): What outputs pass, and what gets blocked or flagged before the user sees them?

Actions (Stage 4): What is the system permitted to do, what requires human approval, what gets logged, and who gets paged when something goes wrong?

If your PRD answers all four with specifics, you’ve thought about this seriously. If it answers two with specifics and hand-waves the other two, you’re shipping a 2024 stack in 2026. That’s the gap I see most often.

The Reframe That Matters

Most PMs treat guardrails as the thing slowing them down. The legal-compliance-paranoia layer that makes shipping harder. Something to minimize so the team can move fast.

That framing is backward.

I write this as someone shipping AI products too. The 2024 stack I wrote isn’t holding up under what I’m building today. None of us has this fully figured out. We are working through them in real time.

Teams without strong guardrails don’t ship faster. They ship once, have an incident, and then spend three months rebuilding trust, answering regulators, and apologizing in public. The shipping-fast companies in 2026 are the ones with the strongest stacks, not the weakest.

Guardrails aren’t the restriction. They’re why you can ship at all.

This isn’t about being cautious. It’s about being trustworthy enough that the people with the budget, users, and data are willing to put AI products in their workflows. No trust, no adoption. No adoption, no business. Your four-layer stack is the reason a risk-averse enterprise customer signs the contract.

Risk tolerance doesn’t mean risk blindness. It means you know exactly where the risks are and you’ve built controls around them. That’s what the stack does.

Monday Morning Audit

Here’s what I want you to do with this.

Pull up your current AI feature. Or the one you’re about to ship. Or the agentic thing on your roadmap that your team keeps waving away as “we’ll figure out guardrails later.”

Write one sentence for each of the four layers. What guardrail fires here?

Stage 1 (Before the Model): _____________

Stage 2 (During): _____________

Stage 3 (After the Model): _____________

Stage 4 (Before Actions): _____________

Any layer where you wrote “nothing yet,” “TBD,” or “engineering is handling it” is a gap. Those are your 2024-stack holes.

Close them before the next version ships. Not because guardrails are nice-to-have. Because without them, you’re not actually shipping a product. You’re shipping a liability.

The One-Line Version

If you remember one thing from this post, remember this.

Guardrails aren’t the restriction. They’re why you can ship.

If you built an AI feature before 2025, your guardrails probably cover two layers. Go check. You’ll know immediately which two you missed.

And if you’re writing your first agentic PRD right now, organize the guardrails section by journey stage, not by category. All four stages are your job.

What layer does your team handle best? Which one’s the biggest gap? Drop it in the comments. I’m curious whether the pattern I’m seeing across teams holds across the broader PM audience.

Related reading:

• AI Agents vs. Agentic AI: A PM’s Guide to the Distinction That Actually Matters

• The Builder’s PRD: What to Think Through Before You Touch Any Tool

• Use the Smallest Capability That Works

• AI Architecture Patterns 101: Workflows, Agents, MCPs, and A2A Systems

• The Q1 2026 AI Vocabulary List Every Product Manager Needs

• Guardrails: Understanding and Implementing LLMs with Responsibility (the 2024 post this one updates)

I couldn’t find it. So I wrote it.

These two terms are doing very different jobs, and when you conflate them, you don’t just sound fuzzy in meetings. You scope the wrong problem. You either over-engineer (building a full agent architecture when you needed one additional capability) or under-engineer (adding a tool call and calling it “agentic” when you’ve barely changed anything about the system’s autonomy).

The Core Distinction

“Agentic AI” is a property. It describes how a system behaves: goal-directed, multi-step, capable of taking action in the world without a human approving every move.

“AI agent” is a thing. A discrete software system, intentionally designed and scoped, that expresses agentic behavior.

The relationship is definitional. An AI agent is an implementation that sits at the higher end of the agentic spectrum. But not everything that exhibits agentic behavior is a fully architected AI agent, and not everything called an “AI agent” is actually very agentic.

That second part surprised me when I dug into it. A chatbot with a single web search tool attached is technically “an agent” by some definitions. It has tool access. It can take an action. But the autonomy is shallow. No planning, no memory across sessions, no self-evaluation. It generates text and stops.

Mildly agentic.

A system like Devin (Cognition’s software engineering agent) runs for hours, decomposes ambiguous goals into task sequences, writes code, runs tests, reads error messages, fixes them, and loops until the build passes. It’s deeply agentic. The architecture underneath looks nothing like the chatbot.

Same label. Very different systems.

The Agentic Spectrum: Five Dimensions That Actually Matter

Agentic behavior isn’t a binary. It’s a spectrum defined by five dimensions. Every AI product sits somewhere on each one, and understanding where your product sits is a PM scoping decision, not an engineering one.

There’s no single “official” taxonomy here. I use five dimensions because they’ve been the most useful in my own work scoping AI systems as a PM.

Every AI product sits somewhere on each one, and understanding where your product sits is a PM scoping decision, not an engineering one.

Here’s the model: five dials you can turn up or down.

1. Tool Use

This is the one most PMs grasp immediately. The model stops describing what should happen and actually does something. It calls a calculator, hits a search API, queries a database, creates a calendar event.

Under the hood, this works through function calling (OpenAI’s term) or tool use (Anthropic’s term). The model outputs structured JSON specifying which tool to call and with what parameters. The runtime executes it, returns the result, and the model folds that into its next output.

A model with one tool is barely agentic. But tool use is the prerequisite for everything else on this list. And the moment you add it, your failure modes change completely. Your product can now take actions in external systems. Rate limits, error handling, rollback capabilities. None of that exists in a pure chatbot. All of it lands on your plate the moment you flip this dial.

2. Planning and Sequential Reasoning

This is where it gets interesting. And where I teach my students to pay close attention.

When I walk through agent architectures in class, planning is the concept that gets the most “wait, really?” reactions. Not tool use. Not memory. Planning. Because this is where the system stops feeling like autocomplete and starts feeling like it’s thinking.

Instead of responding to a single prompt, the model decomposes a goal into subtasks, figures out the order, and executes them step by step. The mechanism behind this is the ReAct pattern (Reason + Act), a widely used agent paradigm introduced in a 2022 Google Research/Princeton paper that combines reasoning traces with tool calls. The model thinks (”I need to find flight options first, then check calendar conflicts, then compare prices”), acts (calls the tool), observes the result, and loops.

Thought → Action → Observation → Thought → Action → Observation → ... → Final Answer

Without planning, you have a chatbot that can use tools. With planning, you have something that pursues a goal across multiple steps without the user guiding every transition.

That’s a meaningful architectural difference. Most agent frameworks you’ll encounter (LangGraph, LangChain, CrewAI) are essentially ReAct or variants of it, wrapped in tooling.

Here’s the PM catch, and this is the part that trips up my students every time: planning introduces non-determinism. Two identical inputs can produce two different execution paths. That means your evals need to test outcomes, not sequences. You can’t diff the expected path against the actual path. You have to ask: did the agent get to the right answer? That’s a fundamentally different testing philosophy than most PM teams are used to, and it changes how you write acceptance criteria, how you define “done,” and how you build confidence that the thing works.

3. Memory

I’ve watched students design agents in class and memory is consistently the thing they underestimate. They get tool use and planning right, but then the agent forgets everything between sessions. A sales copilot that loses account context mid-thread. A support agent that re-asks what the user just told it.

There are four memory types worth knowing.

1. In-context is the current conversation window, fast but temporary.

2. External memory is a vector database or key-value store the agent reads from and writes to. Most production agents combine these two.

3. Semantic memory is what’s baked into the model’s trained weights, separate from any external vector store you set up.

4. Episodic memory is the agent’s record of what it’s already done, separate from its in-context conversation.

The PM question here is a privacy question: what does the agent store? Who can access it? How long is it retained? These are product requirements, not engineering details.

4. Self-Correction

A reactive system does what it’s told and returns the result.

An agentic system asks: was that good enough?

After completing a step, the agent evaluates its own output. Did my code compile? Does this response actually answer the question? If not, it re-enters the planning loop. Some systems use a separate “critic” model for this. Others self-evaluate, which works but introduces blind spots.

The PM implication is latency variance. A task that takes 3 seconds on the first pass might take 45 seconds after two correction cycles. Design for the variance, not the average.

5. Human-in-the-Loop (HITL) Reduction

This was the dimension I had the least clarity on before this research pass. Honestly, I think it’s the most misunderstood one.

Human-in-the-loop isn’t a binary. It’s a dial. A system that drafts an email for your review is mildly agentic. A system that drafts, addresses, and sends it is more agentic. A system that monitors your inbox, identifies which emails need responses, drafts them, and sends them within a confidence threshold is deeply agentic. Same capability, three very different levels of autonomy.

Each step along that dial introduces more risk and more product design complexity. Where you set it is a product decision, not a technical constraint.

The question to ask before removing any human checkpoint: what’s the worst-case output if the agent is wrong here? For low-stakes outputs, low HITL is fine. For high-stakes outputs, every checkpoint you remove needs a corresponding increase in confidence thresholds and rollback capability. Sending emails to customers, executing transactions, modifying production databases. Those aren’t places to discover your confidence threshold was wrong.

So What Is an AI Agent, Technically?

An AI agent is a system architected to score high across all five dimensions: it uses tools, plans multi-step sequences using something like ReAct, maintains memory across steps, evaluates its own outputs, and operates with limited human checkpoints.

Here’s how the architecture breaks down:

• Perception layer: Receives inputs from the environment (user messages, API responses, database reads, tool outputs)

• Memory layer: Short-term (in-context) and long-term (vector DB or key-value store)

• Planning and reasoning: The ReAct loop, often powered by a capable frontier model

• Action layer: Tool calls, API integrations, function execution

• Evaluation layer: Self-scoring or critic model, feeds back into planning

The Think → Plan → Act → Observe loop connects all of these. The agent observes its environment, thinks about what to do, plans a sequence of actions, acts, observes the result, and loops until the goal is satisfied or a stopping condition is met.

A concrete example: a support agent that reads incoming tickets (perception), retrieves past conversations and account history (memory), plans a response with possible actions (planning), issues a refund or updates a record (action), and checks for policy compliance before sending (evaluation). Each layer is doing a distinct job. Remove one and the system degrades in a specific, predictable way.

This is meaningfully different from a chatbot with a search plugin. The architecture complexity, the failure surface, and the design requirements are all different.

The Product Question Nobody Asks

When someone on your team says, “We should build an AI agent for this,” the right follow-up isn’t “What framework should we use?” It’s: how agentic does this actually need to be?

Map the requirement to the five dimensions:

• Does it need to use tools? → Add tool use.

• Does it need to execute a multi-step sequence? → You need planning.

• Does it need to remember the state across sessions? → You need external memory.

• Does it need to improve its own outputs? → You need evaluation loops.

• Does it need to run without human review? → You’re deliberately reducing HITL. Design for that explicitly.

Most products need two or three of these, not all five. Most “AI assistants” today sit at the tool-use plus light planning level. Systems like Devin have all five dials turned up. The gap between those two things is enormous in cost, complexity, and failure surface.

The Use the Smallest Capability That Works principle applies directly here. More agentic doesn’t mean better. It means greater complexity, a larger failure surface, greater latency variance, and more privacy obligations. You earn each dimension by showing that the simpler version can’t do the job.

The Interview Version

If someone asks you “what’s the difference between AI agents and agentic AI?” in an interview, here’s where I’d land.

The 30-second answer:

“Agentic AI describes a property of AI behavior: how goal-directed, multi-step, and autonomous a system is. It’s a spectrum, not a category. AI agents are systems deliberately architected to sit at the higher end of that spectrum, combining tool use, planning, memory, and self-correction into a persistent loop. As a PM, I think about agenticism across five dimensions, and the product question I always ask is: how far along each dimension does this use case actually need to go? Because each dimension adds complexity, failure modes, and design requirements that a simpler system avoids.”

If they give you more time, extend it with an example. A simple RAG chatbot has one dial turned up. Devin has all five. That contrast does a lot of work in a short window. It signals you understand the architecture isn’t theoretical. It’s a set of deliberate product and engineering decisions that compound.

What the interviewer is actually testing.

This question isn’t about whether you memorized the definition. It’s about four things:

• Technical depth. Do you know what’s under the hood? Tool use, ReAct, memory patterns.

• Scoping judgment. Do you reach for “full agent” every time, or do you ask how much autonomy the use case actually needs?

• Appropriate skepticism. Do you push back on vendor marketing that labels every chatbot with a tool call an “agent”?

• Business context. Can you connect agenticism to cost, latency, failure surface, and user trust?

A textbook definition nails the first one and misses the other three. That’s the gap between a candidate who sounds smart and a candidate who gets the offer.

That’s the answer. But here’s what separates candidates who get the offer from candidates who don’t. And it has nothing to do with the definition itself.

Read more

Claude Code has it. Cursor has it. bolt.new has it. The idea is the same across all of them: before the AI writes a single line of code, stop and align on the approach. What files to modify? What components to use? How the pieces fit together.

It’s a good idea. And it’s not enough.

Plan Mode asks: How should I build this?

That’s an implementation question. It assumes you’ve already answered the harder questions. What problem are you actually solving? What does the user experience at each step, not just at the feature level? What happens when step 3 gets a null value? Where does a human need to stay in the loop?

Those questions don’t live in Plan Mode. They live in the thinking you do before you open any tool. And if you skip them, Plan Mode just helps you build the wrong thing faster.

In my last post, What n8n Teaches You That Claude Code Doesn’t, I wrote about why building visibly in n8n changes how you think about systems. This is the follow-up: the mental model n8n forced me to develop, written down so you can apply it in any tool.

I’m calling it the Builder’s PRD. Not because it’s a document. Because it’s the thinking an AI builder needs to do before any build starts.

What You’ll Learn (9-minute read):

• Why Plan Mode isn’t enough and what it’s missing

• The 5 foundational questions you already know (but AI tools make you forget)

• The 3 questions AI building specifically demands, and that Plan Mode can’t help you with

• Why this isn’t just for PMs anymore

Why Plan Mode Isn’t Enough

Here’s what Plan Mode is good at. You describe what you want. The AI reads your codebase, proposes an approach, shows you what it’s going to do before it does it. You approve. It builds.

That’s genuinely useful. It prevents the “one-shot prompt and pray” problem. You’ve effectively briefed a developer before they start.

But notice what it doesn’t ask.

It doesn’t ask whether you actually need AI at this step, or whether a simple rule would do. It doesn’t ask what the user is experiencing at this point in the flow, or whether they’ve earned enough trust in the system to hand control over. It doesn’t ask what happens when the model is wrong, or what it costs the user when it is.

Plan Mode starts where the Builder’s PRD ends. It’s the contractor reviewing blueprints. The Builder’s PRD is the architect’s work that produced them.

The 9 Questions

Nine questions, two tiers. Answer them before you open any tool.

The first five are PM fundamentals. You’ve heard them before. I’m including them because AI builder tools make it dangerously easy to skip them, and when you skip them with AI, the consequences show up faster and uglier than they do in traditional product development.

The last three are the questions AI building specifically demands. These don’t exist in a world without LLMs, agents, and generated outputs. They’re the reason this post exists.

Read more

Then I tried it. And then I built a whole workflow with it live at a PDMA Seattle event, which I wrote about here. That post is about what I built. This one is about what n8n taught me that no other tool has.

I was right that it would make me slow down. That turned out to be the point.

What You’ll Learn (7-minute read):

• Why Claude Code is the wrong starting point if you actually want to understand what you’re building

• What n8n forces you to confront, node by node

• How I use Claude as a build collaborator (not a vending machine)

• The capability ladder in practice, not just in theory

• When to graduate to Claude Code — and why the sequence is the whole point

What Claude Code Hides (And Why That’s a Problem for PMs)

Claude Code is genuinely good. I use it. I’m not here to argue against it.

But here’s what happens when you build with it as a PM. You describe what you want. Code appears. You run it. Something works. And then you move on, because it works, and you’re busy.

The gap between “it works” and “I understand why it works” doesn’t feel urgent when things are going well. It becomes very urgent when something breaks in production and you’re staring at a codebase you didn’t write, trying to explain to an engineer where you think the problem is.

You shipped a feature. You just don’t understand the feature you shipped.

I started calling this transcribing. You’re not building. You’re taking dictation from a model and calling it a build.

What n8n Forces You to See

In n8n, your workflow lives on a canvas. Every step is a node. Data flows from one node to the next, and you can click into any node and see exactly what went in and what came out.

Nothing is hidden. That’s uncomfortable at first.

If you’re calling an API, you configure that connection yourself. If you’re transforming data, you write the expression. If you’re passing something to an LLM, you construct the prompt, decide what context to include, and watch exactly what the model receives.

When I built my first real workflow in n8n, I hit a step where I needed to route a message based on a field value. My instinct was to use the AI node. I’d been conditioned by months of Claude Code to reach for the model when something felt even slightly ambiguous.

I stopped. A switch node handled it in about 20 seconds.

That moment is hard to manufacture in Claude Code, because Claude Code doesn’t create that pause. It just writes the thing you asked for. n8n makes you decide, at every step, what kind of capability you actually need. That friction is a feature.

n8n Is System Design Practice

There is an interview in technical product management interviews called the system design interview. You’d get a whiteboard and a prompt, “design a URL shortener,” “design a notification system”, and you had to walk through how the pieces connected. What talks to what? Where does data live? How does the system handle load, failure, and scale?

Most PMs prepared for these by studying other people’s diagrams. Which worked, sort of, until the interviewer asked a follow-up question and you realized you’d memorized a picture without understanding its reasoning.

n8n is the version of that practice where you actually have to build the thing.

Every workflow you construct in n8n is a system design exercise. You define the inputs. You decide what transforms the data and where. You choose what calls an external API and what stays internal. Think about what happens when a node fails. Does the workflow stop? Retry? Route to a fallback? Those aren’t configuration questions. They’re the same questions a good system design interviewer is trying to get at.

The difference is you can’t fake it in n8n. You can sketch a vague architecture diagram and talk your way through the gaps. You cannot place a node that doesn’t actually connect to the previous one. The tool forces precision in a way that whiteboarding doesn’t.

If you’re preparing for AI PM interviews or just trying to get sharper on how AI systems actually fit together, building in n8n is more useful than any prep guide I’ve seen.

The Capability Ladder, Finally Making Sense

I’ve written about the capability ladder before. Rules → ML → small language models → LLMs → agents → multi-agent systems. Start at the bottom. Only move up when the rung below genuinely can’t do the job.

Knowing that principle and feeling it are two different things. n8n is where I felt it. When you’re placing nodes manually, you can’t skip steps. You have to decide, for each one: is this a rule? A condition? A classification? Or does this actually need a language model?

Most of the time, for the workflows I was building, the answer was further down the ladder than I expected. Checking if a field is empty doesn’t need GPT-4. Routing based on a known value doesn’t need an LLM. Formatting a date really, really doesn’t need an LLM.

Every unnecessary model call costs money and adds latency. In Claude Code, those decisions are buried in generated code you didn’t write and probably won’t audit. In n8n, they’re sitting right there on the canvas.

If you’ve read my post on Use the Smallest Capability That Works, this is what that principle looks like when you actually build with it, not just think about it.

How I Use Claude Inside n8n (This Part Changed How I Work)

Once I was building node by node, I started using Claude differently.

Instead of “build me this workflow,” the prompt became: I’m building a workflow that does X. I’m at this specific step. The input coming in looks like this. What should this node do, and what should the output look like?

One node. One question. One answer.

This sounds slower. It is, marginally. But it means I understand every single thing I put on that canvas. When something breaks, I know where to look. When I hand it to an engineer to scale, I can explain the logic because I built the logic, not because I read it off generated code.

I also started using Claude as a thinking partner on capability decisions. When I’m designing a node that involves something like intent classification or deciding where to route a user message, I’ll ask: Is this the right capability for this step? What happens when the model gets it wrong? Where does a human need to be in the loop?

That’s a PRD conversation happening inside a build session.

Then Go to Claude Code

None of this is an argument against Claude Code. It’s an argument about sequence.

Once you understand how a system works where data flows, what each step is actually doing, where AI earns its place, and where it doesn’t, code-first tools make you significantly more capable. You know what you’re asking for. You can review what gets generated. When something breaks, you have a mental model.

The PMs who struggle most with AI builder tools are the ones who started with code generation. They can ship things. They can’t explain what they shipped. When something goes wrong, they’re stuck.

The ones who built a few clunky n8n workflows first, who cursed at node configurations and figured out why their data was the wrong shape, move to Claude Code and suddenly they’re dangerous. The code is just an expression of a system they already understand.

Claude Code is not the problem. Starting with Claude Code, before you understand what you’re building, is the problem.

n8n puts the system design on the canvas. You see the data move. You make an active decision at every step about which capability you need. You build slowly enough to actually understand what you’re building.

The mental models you develop in n8n outlast any specific workflow you build. That’s the thing I didn’t expect when I finally stopped avoiding it.

Start there. Graduate to code. Know what you’re looking at either way.

What was the first thing n8n made you see that you’d been missing? I’m curious whether the capability ladder moment lands for others the way it did for me, or whether it’s something else entirely.

Related reading:

• Use the Smallest Capability That Works

• AI Architecture Patterns 101: Workflows, Agents, MCPs, and A2A Systems

• To Agent or Not to Agent: That’s the AI Product Question

I paused. Wait, what? Did I hear that correctly?

My genuine first reaction was that Claude’s API had hallucinated again. Made up a term that sounded impressive but didn’t exist. I’ve caught it doing that before.

But I searched it. And it was real. And the more I read, the more I realized this was a concept I needed to understand properly, not just nod along to.

So I did my research. This post is what I found.

What you’ll learn (9-minute read):

• Why “world model” means three different things right now (and which one you’ll actually encounter)

• The core building blocks: perception, dynamics, and planning

• How world models differ from LLMs and agents, and how they fit together

• Where world models are showing up in real products today

• What changes for AI PMs when world models enter the stack

• A practical Phase 1/2/3 roadmap for thinking about this in your product

The Terminology Problem

“World model” is one of those terms where you can spend 30 minutes searching and come out more confused than when you started. I’ll save you that experience.

The term means three different things depending on who’s using it.

Yann LeCun’s framing: A learned internal representation of how the world works. Physics, causality, 3D space, object permanence. The things humans internalize from infancy. LeCun (Meta’s Chief AI Scientist) has spent years arguing that this is the missing ingredient for human-level AI.

Video generation framing: Models like Sora (no longer an OpenAI offering) are called “world models” because they’ve apparently learned enough about physical dynamics to generate realistic video sequences. This is contested. Some researchers argue that predicting pixels isn’t the same as understanding physics.

Robotics and simulation framing: A differentiable simulator that an agent can use to run rollouts before taking real actions. This is the most concrete version and the one with the most immediate product relevance.

These are related but not identical. For this post, I’m focused on the third framing. World models as a simulation and planning infrastructure. The others are worth knowing about (I covered the LeCun framing in my Q1 2026 AI Vocabulary List), but if you’re building products, the simulation framing is where this gets real.

What a World Model Actually Does

Plain language: a world model is an internal simulator that lets an AI predict how the world will change before it acts.

Here’s where the concept clicked for me.

A warehouse robot needs to navigate around an obstacle to reach a package. Without a world model, the robot tries a path. Bumps into something. Backs up. Tries again. Learning from interaction. That works, but it’s slow, it causes wear, and in a hospital corridor or factory floor, it can be unsafe.

With a world model, the robot simulates 10 possible paths before it moves a single motor. It predicts which paths hit obstacles, which ones are fastest, which ones minimize risk. Then it picks the best one and moves.

That’s the core capability. Planning in imagination instead of only learning from experience.

A bit more technical: a world model takes a current state and an action as inputs, and predicts the next state. Sometimes it also predicts a reward signal. This lets an agent plan across sequences of actions, not just react to what’s in front of it.

How World Models Work: Three Building Blocks

Most world model architectures have three components. I’ll walk through each one using the warehouse robot, because the abstraction makes a lot more sense with a concrete example running through it.

Perception: What’s Happening Right Now?

The robot is getting camera feeds, LiDAR scans, and sensor readings simultaneously. That’s a massive amount of raw data. Perception converts all of it into a compact representation of “what’s going on right now.”

For the warehouse robot, that means taking the camera feed showing boxes on shelves, a forklift blocking aisle 3, and a worker walking through aisle 5, and compressing it into a state representation that the rest of the system can work with. In ML terms, it encodes the environment into a latent state. The goal: capture what matters without carrying every raw pixel.

A quick note on computer vision vs. world models:

Computer vision answers: What am I looking at? A world model answers: what happens next if I do X, and which X is best?

In our warehouse robot, computer vision is the part that says “forklift in aisle 3, worker in aisle 5.” That’s perception. The world model runs 100 simulated futures for each of the following: if I turn left, if I go straight, and if I wait 3 seconds. Computer vision is the eyes. A world model is the eyes, plus imagination, plus judgment.

Dynamics: What Happens Next?

This is the world model proper. It takes the compressed state from perception and predicts what happens next, given an action. If the robot turns left, what does the world look like in 500 milliseconds? If it speeds up, does it clear the forklift before the worker crosses?

This is where the physics, object permanence, and environmental behavior get encoded. A good dynamics model generalizes: it can predict states it never saw in training, not just memorize transitions. That generalization is what makes simulation useful. Without it, you’re just replaying recorded scenarios.

Planning: Which Path Is Best?

The planner uses the dynamics model to look ahead. Run 100 simulated rollouts. Score each one. Pick the action sequence that leads to the best outcome.

For the warehouse robot: simulate 100 possible routes in 200ms of compute time. Three routes hit the forklift. Twelve routes cross the worker’s predicted path. The planner picks the route that reaches the package fastest while minimizing collision risk. All of that happens before the robot moves an inch.

Compare that to 45 minutes of physical trial and error. That’s the business case for world models in a single number.

If you’ve read my AI Architecture Patterns 101 post, you’ll recognize the Think → Plan → Act → Observe loop for agents. World models make the “Plan” step dramatically more capable. Right now, most agents plan by reasoning over text and tool descriptions. With a world model, they plan by simulating what will actually happen.

Different categories of capability.

Where World Models Fit: LLMs, Agents, and World Models

Read more

That list changed a lot over the last six months. Some terms I’d been tracking for a while finally became unavoidable. Others showed up out of nowhere and were suddenly everywhere. A few I’d never heard six months ago are now part of how I think about AI product management.

The vocabulary changed because the behavior changed or new research emerged. We went from AI that answers to AI that acts. And then, quietly, AI that understands the physical world.

Here are the terms I’d keep in mind as a product manager this Q1 of 2026. I’ve grouped them into five clusters. The first one is the furthest from most PMs’ current work. I’m starting there anyway, because it’s the one that tends to surprise people, and I think the surprise is worth having early.

Cluster 1: Physical and Spatial AI (The Horizon Terms)

You might not need these for your next sprint. You’ll probably need them in your next couple of annual planning cycles.

Start with world models, because everything else in this cluster builds on them. A large language model predicts the next token. A world model predicts the next state of the world. It learns physics, gravity, spatial relationships and object permanence. Not from rules someone programmed, but from watching how the world actually behaves.

Yann LeCun put world models at the center of his argument at the AI Action Summit in Paris in early 2025, claiming they’re better suited for human-level AI than today’s LLMs. That claim is still contested. What’s not: the infrastructure is being built. NVIDIA launched Cosmos specifically for world models. Fei-Fei Li’s World Labs released Marble, generating 3D environments from a text prompt. Runway shipped world-model-style video generation in late 2025.

Not research demos. Products.

The PM question: if you’re building anything involving a physical environment (robotics, industrial simulation, AR/VR, gaming), world models are approaching your product surface. The question isn’t whether to care. It’s when.

Three related terms build on this. Physical AI is what world models power: any physical process learning from and applying AI (Citi Research’s definition). The robots at GTC, autonomous vehicles training on synthetic scenarios, surgical systems rehearsing procedures before touching a patient. Spatial intelligence is Fei-Fei Li’s framing for the capability underneath: not recognizing what’s in an image, but understanding depth, relationships, and consequences. “That’s a cup on a table” vs. “if I push it two inches left, it falls.” Embodied AI is the broadest category: AI that interacts with the physical world through sensors and actuators.

These aren’t replacing LLMs next quarter. But the companies building physical products are already in this vocabulary. And they’re most likely hiring PMs who understand it.

Cluster 2: Agentic Architecture

This is the cluster I hear most about right now. In interviews, in sprint reviews, in product discussions. The terminology is evolving fast because the architecture is still being figured out in real time.

If you only learn one term from this cluster, make it agentic workflow: AI that executes multi-step tasks across tools (email, CRM, code repos, files) with minimal human input between steps. Chatbots answer. Agents act. That’s the line. And every time you cross it, the first product question isn’t “what should the agent do?” It’s “where does a human stay in the loop, and what happens when the agent gets it wrong?”

Orchestration is where the real architecture decisions live. It’s the coordination layer managing which agents fire in what order, how they hand off, and when to escalate. You’re not just speccing the model. You’re speccing the flow, the fallbacks, and the error states.

Two terms you’ll see together: Model Context Protocol (MCP) and Agent-to-Agent (A2A). MCP is Anthropic’s open standard for how agents connect to tools and data sources in a structured way. Adoption has been fast. Within a year of launch, MCP integrations became a default part of most agentic toolchains. A2A is the pattern in which specialized agents collaborate and hand off work to one another. When you’re scoping a complex agentic product, “one agent or many?” is a real architecture decision with trade-offs in debuggability, latency, cost, and failure isolation.

Here’s the term that surprised me most when I dug into it for my agents vs. agentic AI post - coming in April: Human-in-the-loop (HITL) isn’t new, but its meaning shifted. In 2022, HITL meant that a human reviewed the model’s output before it shipped. In 2026, it means a human approves specific actions before an agent takes them. That’s a different design problem. Where do you put the checkpoints? How do you surface them without making the agent useless? What triggers escalation vs. silent failure? Every agentic PRD needs an explicit answer.

Two more for your production vocabulary. AgentOps is the emerging discipline for managing agents in production, parallel to DevOps and MLOps. Analyst firms like Deloitte expect enterprises to spin up dedicated AgentOps-style functions as they scale agents in production. Observability goes deeper: not just monitoring what an agent did, but tracing why it decided to do it. LangChain’s 2025 State of Agent Engineering report found that the vast majority of organizations running agents in production had implemented some form of observability. Without it, you’re debugging failures from logs that show what happened, but not how the reasoning got there.

For PMs, observability is a launch requirement. Not a nice-to-have.

Cluster 3: Reasoning and Computing

These terms are showing up in infrastructure conversations, pricing discussions, and model selection decisions. If you’ve been nodding along without fully understanding them, this section is for you.

Test-time compute changed how I think about pricing for AI features. Here’s the shift: traditional model scaling was about training. More data, more parameters, better model. Test-time compute is about what happens at answer time. The model gets more computing power to think through a problem before responding, exploring multiple reasoning paths, and checking its own work.

The PM implication is direct: better answers increasingly require more inference. When your team says, “We’ll use the reasoning model for this feature,” that’s a cost decision as much as a capability decision. Pricing, margins, and latency all depend on how much thinking you allow per query.

Reasoning models are built specifically for this tradeoff. Instead of pattern-matching to a quick answer, they generate intermediate steps, verify them, and revise. A standard completion might take 3 seconds. A reasoning query on a complex problem can take 60 seconds and generate thousands of internal tokens the user never sees. Right-sizing, which queries get a reasoning model vs. a standard one, is a real product decision. Extended thinking is the user-facing name for the same concept. Whether to surface the “thinking” state to users or hide it is a UX question with no obvious right answer.

Two infrastructure terms are worth knowing. Mixture of Experts (MoE) is the architecture that makes massive models affordable to run. Instead of activating all parameters for every query, a MoE model activates only the relevant subset. A trillion-parameter model might activate 20 billion for any given input. Distillation compresses a larger model’s behavior into a smaller one. The big model teaches the small model through outputs, not weights. When you hear “we can run this on-device” or “we cut GPU cost by 70%,” distillation is usually the reason.

The framing I keep coming back to: frontier vs. efficient model classes. 2026 is splitting into two lanes. Frontier models pushing capability at enormous compute cost. Efficient models optimized for modest hardware. IBM researchers put it plainly: “We can’t keep scaling compute, so the industry must scale efficiency instead.” For PMs, this changes the build vs. buy analysis. Efficient open-source models are getting good enough for use cases that previously required frontier APIs.

Cluster 4: Context and Memory

I wrote about context windows in March 2025. The vocabulary has gotten more specific since then.

Context engineering is the term to know here, and it’s not prompt engineering. Prompt engineering is about what you ask the model to do. Context engineering is about what the model knows when it answers: what you select to include, what you compress, what you isolate per task and what you store across turns. As context windows expanded to hundreds of thousands of tokens, decisions about what to fill them with became product-architecture decisions.

Two supporting terms. KV cache (key-value cache) is the internal memory store that holds processed token representations so the model doesn’t reprocess them every turn. You won’t configure this directly, but when your infrastructure team talks about “cache hit rate” or “cache miss scenarios,” this is what they mean. It drives latency and cost in long conversations. Long-term memory is external storage that lets an agent remember things across sessions, not just within a single context window. What gets stored? Who sees it? How does the user delete it? How does the agent decide when to retrieve vs. ignore old memories?

Product questions. Not engineering ones.

Cluster 5: Building and Governance

Vibe coding is the one I have the most personal experience with. I built a family Guess Who game in Bolt. 34 versions. Collins Dictionary named the term its 2025 Word of the Year. Andrej Karpathy coined it: describe what you want to an AI coding assistant, iterate through natural language instead of writing syntax. The practical takeaway for PMs: prototyping cycles got dramatically shorter. What took a week of engineering mockup time now takes an afternoon of prompt iteration.

The counterargument worth hearing: there’s a real “vibe coding hangover” as teams discover that code written fast is hard to maintain. The wins are genuine for prototypes and internal tools. The risks are real when vibe-coded artifacts become production dependencies without proper review.

Prompt injection becomes critical the moment your agents read external content. An adversary hides malicious instructions inside a document, email, or webpage. The agent processes them and acts on them rather than pursuing its original goals. In a chatbot, the worst case is a weird response. In an agent that sends emails, executes code, or modifies data, it’s a security threat. Every agentic product that ingests external content needs an explicit answer to: “What happens if something we ingest tells the agent to do something it shouldn’t?”

Guardrails have the longest evolution of any term on this list. I wrote the original guardrails post in February 2024. The framing then was accurate for that moment: output filters. Block toxic content. Flag hallucinations. Enforce length limits. A single layer, mostly at the response time.

The meaning expanded. A lot.

In 2026, guardrails means a full stack: who can deploy a model change, who approves new tools an agent can access, how prompts are versioned, what gets logged and what triggers a human review. The concern shifted from “what does the model say” to “what does the system do.” Because agents take actions, a guardrail failure now has operational consequences. An agent that miscalibrates and exports a customer list is an incident, not a bad response.

If you’re writing a PRD for any agentic feature, the guardrails section needs to answer: what the agent is permitted to do, what requires human approval, what gets logged, and who gets paged when something goes wrong. AI governance stack sits above all of this: the organizational layer that owns AI policy, approves model integrations, and handles behavior changes in shipped products. As a PM, knowing who those owners are is part of the job now.

Know Now vs. Know Soon

This is my honest read on which clusters are active vocabulary and which are horizon vocabulary.

Know now (your team is using these words): everything in Clusters 2, 3, and 5. Agentic architecture, reasoning models, observability, vibe coding, prompt injection, guardrails. If these aren’t in your working vocabulary yet, they should be.

Know soon (you’ll need these within 12-18 months, depending on your industry): everything in Cluster 1. World models, physical AI, spatial intelligence. If you’re in consumer software, you have time. If you’re anywhere near robotics, manufacturing, healthcare devices, or autonomous systems, you might already be late.

Know enough to ask smart questions (Cluster 4): context engineering and agent memory matter a lot if you’re shipping agentic products. If you’re not, understanding the concepts is enough for now.

The vocabulary shifted because the technology shifted. AI moved from answering to doing. And now the earliest parts of it are starting to move from doing in digital space to doing in physical space.

Knowing these terms won’t make you a better PM by itself. But not knowing them means you’ll lose time in every conversation where they come up, either asking for a translation or (worse) not asking and missing the implications.

Part of what separates PMs who advance in AI roles from those who plateau is frameworks and judgment. A real part of it is vocabulary. You can’t push back on a technical decision you don’t have words for.

Related posts:

• Understanding Context Windows — the original deep dive on how context limits affect your product

• AI Architecture Patterns 101 — workflows vs. agents vs. MCP vs. A2A

• Guardrails: Understanding and Implementing LLMs with Responsibility — the 2024 post this one updates

Sources:

• Built In: “World Models Are the Next Big Thing In AI”

• Scientific American: “World models could unlock the next revolution in artificial intelligence”

• The New Stack: “AI Engineering Trends in 2025: Agents, MCP and Vibe Coding”

• IBM Think: “The trends that will shape AI and tech in 2026”

• LangChain State of Agent Engineering 2025

• Vibe Coding (Google Cloud)

• Elektor Magazine: “2026: An AI Odyssey — The Vibe-Coding Hangover”

• Deloitte: “More compute for AI, not less”

• IBM: “What is AgentOps?”

I’ve answered this question dozens of times, and the thing that sharpened my thinking the most was realizing it’s actually two different questions. The build vs buy calculus is fundamentally different for traditional ML vs GenAI, and once you make that distinction, the whole framework clicks.

Once I started teaching this topic in my classes, my thinking sharpened further. Here’s the framework I use now.

What You’ll Learn (8-minute read)

• Why build vs buy is a completely different question for traditional ML vs GenAI

• A decision framework for each type

• The hidden costs most people forget

• How to answer when you don’t know which type the interviewer means

• Follow-up questions to prepare for

The Distinction Most People Miss

When interviewers ask about build vs buy for AI, they might be thinking about two very different scenarios.

Traditional ML means prediction tasks specific to your business. Fraud detection, demand forecasting, churn prediction, recommendation engines, and image classification for your particular use case.

GenAI means language and content tasks powered by foundation models. Chatbots, summarization, content generation, code assistance, and document analysis.

The “buy” options look completely different for each. And that changes everything about the decision.

In traditional ML, “buy” often means purchasing a vertical SaaS solution or using an AutoML platform. But there may not be a pre-built solution for your specific prediction problem. Your fraud patterns aren’t the same as another company’s fraud patterns.

For GenAI, “buy” means using foundation model APIs like GPT-4, Claude, or Gemini. These are genuinely general-purpose and work surprisingly well out of the box.

This is why I always clarify which context the interviewer means before diving into my answer.

Traditional ML: Build Is Often the Default

Here’s the thing about traditional ML that took me a while to internalize: your prediction task is often inherently unique to your business.

Your customer churn signals depend on your specific product and user base. Your demand forecasting needs to account for your supply chain, your seasonality, and your market dynamics. There’s no “churn prediction API” that knows your customers.

This means the “just use an API” option that works so well for GenAI often doesn’t exist for traditional ML.

When to use pre-built solutions for traditional ML:

Your problem is genuinely common. Email spam filtering, basic sentiment analysis, and standard OCR. These are solved problems with commodity solutions.

Vertical SaaS exists for your industry. Fraud detection platforms for fintech, diagnostic tools for healthcare. These work because they’ve seen patterns across many similar companies.

AutoML platforms can handle your task. If you have structured tabular data and a standard classification or regression problem, tools like Google’s Vertex AI or Amazon SageMaker can get you a decent baseline quickly.

You need a baseline before investing in custom work. Sometimes you want to validate the opportunity first.

When to build custom models for traditional ML:

Your prediction task is specific to your business. Your fraud patterns, your churn signals, your demand drivers.

Pre-built solutions don’t exist or perform poorly on your data. You’ve tried the off-the-shelf options, and they don’t work.

The model is core to your differentiation. If predictions are the product, you probably need a custom solution.

You have sufficient labeled data. Traditional ML needs thousands to millions of labeled examples, depending on complexity. No data, no model.

GenAI: Buy Is Usually the Default

GenAI has a fundamentally different dynamic. The “buy” option is remarkably capable.

Foundation models like GPT-4 and Claude are trained on massive datasets and designed to be general-purpose. They can summarize, answer questions, generate content, analyze documents, and write code without any customization. I covered the technical foundations in my posts on how LLMs actually work and foundation models 101.

This flips the default. Start with existing APIs and only build custom when you have clear evidence it’s necessary.

When to use existing APIs for GenAI:

You need general language capabilities. Summarization, Q&A, generation, analysis. Foundation models handle these well.

Prompting and RAG can achieve your goals. I covered this hierarchy in my RAG vs fine-tuning post. Most production GenAI systems use prompting plus RAG. Few need fine-tuning.

You don’t have unique data that would meaningfully improve a model. Having data isn’t the same as having data that creates differentiation.

Speed to market matters. APIs let you ship in days. Custom models take months.

When to consider custom models for GenAI:

You have genuinely proprietary data that creates a competitive advantage. Not just “we have data” but “we have data no one else has that would make our model meaningfully better.”

Your use case requires capabilities existing models can’t provide. You’ve tried prompting, tried RAG, and still can’t get there.

Latency or cost at scale makes APIs impractical. At very high volume, the economics might favor running your own models.

Regulatory requirements demand on-premise deployment. Some industries can’t send data to third-party APIs.

The key difference: for GenAI, the burden of proof is on building. You need evidence that the default (APIs + prompting + RAG) isn’t working before investing in custom models.

The GenAI Customization Hierarchy

For GenAI, there’s a progression of options I walk through when teaching this:

Level 1: Prompting. Write good system instructions, provide examples, and specify output format. This gets you surprisingly far. Always start here. I covered the fundamentals of prompting in my prompting post.

Level 2: RAG. Give the model access to your specific documents by retrieving relevant context and injecting it into the prompt. This solves most “the model doesn’t know about our stuff” problems.

Level 3: Fine-tuning. Train the model further on your data to change its behavior patterns, tone, or domain vocabulary. This is for when prompting and RAG can’t achieve the behavior you need. I covered this in my fine-tuning post.

Level 4: Custom models. Training from scratch or heavy modification of base models. Requires massive data, compute, and expertise. The bar should be very high.

Most production GenAI systems use Level 1-2. Some use Level 3. Very few need Level 4.

One thing worth noting: “custom model” isn’t binary. There’s a spectrum from parameter-efficient fine-tuning (LoRA, QLoRA) to distillation to continued pre-training to full training from scratch. When someone says they “built a custom model,” they usually mean LoRA fine-tuning or distillation, not training from scratch. Knowing this helps you ask better follow-up questions in interviews.

Hidden Costs Most People Forget

This is where I spend a lot of time in my classes, because the costs extend far beyond the initial training or API fees.

For traditional ML:

Data infrastructure. Collecting, storing, and processing training data. Building feature pipelines. Maintaining data quality.

Labeling. Getting ground truth labels, either from natural feedback loops or manual annotation. This is often the bottleneck and the most expensive part.

Deployment infrastructure. Serving predictions at the latency and scale you need. Often harder than training the model.

Monitoring and retraining. Models degrade as data distributions shift. You need to detect drift and retrain periodically.

For GenAI:

Evaluation complexity. How do you measure whether your fine-tuned model is actually better? GenAI evaluation is genuinely harder than traditional ML metrics.

Alignment and safety. Fine-tuning can break safety behaviors. You may need to re-implement guardrails.

Update cycles. When the base model improves (GPT-4 → GPT-5), do you need to re-fine-tune? You might fall behind the frontier while maintaining your custom version.

Opportunity cost. Your team is working on model customization instead of product features and prompt optimization.

How to Answer When You Don’t Know Which Type They Mean

Read more

I searched it. Still didn’t get it. So I walked over to my ML engineer, lucky for me, he was a friend, and asked him to just explain it like I was five.

He did. And I remember thinking: not everyone has that friend.

This post is that friend.

What you’ll learn (9 minutes):

• What inference actually is

• The two phases every LLM response goes through

• The three metrics that tell you whether your product feels fast or broken

• Why do longer prompts slow things down

• What batching means for your users

• The one tradeoff every AI PM needs to understand before making deployment decisions.

What “Inference” Actually Means

Training is when a model learns. Inference is when a model runs. One request, one response, one inference.

Every time a user sends a message to your AI product and gets a response back, that’s inference. The model takes your input, does an enormous amount of math, and produces output. That’s it.

The reason inference matters for PMs and why it’s worth understanding in some depth is that it drives three things you care about directly:

1. How fast your product feels,

2. How much does it cost to run, and

3. How many users can you serve at once?

Get inference wrong and you’re either burning money or delivering a sluggish experience or both.

The Two Phases of Every LLM Response

Here’s the part most people don’t know. When you send a prompt to an LLM, the model doesn’t process it the way you might imagine, reading it left to right and generating words as it goes. It actually works in two completely distinct phases.

Phase 1: Prefill

The model takes your entire input, your system prompt, conversation history, retrieved documents, and user messages, and processes them all at once. In parallel. This is called prefill.

During prefill, the model reads every token in your prompt simultaneously and builds an internal representation of the whole context. It’s compute-heavy, but fast relative to what comes next because all that parallel processing happens in one shot on the GPU.

The thing that comes out of prefill is called the KV cache (Key-Value cache). Think of it as the model’s compressed memory of everything you just told it. This cache is what the next phase uses.

One critical implication: longer prompts = more expensive, slower prefill. If your system prompt is 3,000 tokens and your RAG pipeline retrieves 2,000 tokens of context, the model has to process all 5,000 tokens before generating a single word of response. That’s why context window management, which I covered in depth in my post on Understanding Context Windows, is directly connected to your inference costs and latency.

Phase 2: Decode

After prefill, the model starts generating your response. One token at a time.

This is the decode phase. And here’s the part that surprises people: it cannot be parallelized. Each new token depends on every token that came before it. The model generates “The,” then looks at “The” to generate “refund,” then looks at “The refund” to generate “policy.” Sequential by design.

Each generated token is appended to the input and fed back into the model to generate the next token. The KV cache from prefill makes this manageable; without it, the model would have to reprocess the entire context from scratch for every token it generates.

Decode is slower, per token, than prefill. But it’s also where your users are actually watching something happen — the text streaming in. Which brings us to the metrics.

The Three Numbers That Tell You If Your Product Feels Fast

Time to First Token (TTFT)

TTFT is how long users wait before they see anything at all. It’s essentially the duration of the prefill phase plus a small amount of overhead.

A chatbot might require a TTFT under 500 milliseconds to feel responsive, while a code completion tool may need a TTFT below 100 milliseconds for a seamless developer experience. If your TTFT is over 3 seconds, users start to wonder whether the product is broken.

The PM implication: anything that makes your prompt longer makes your TTFT worse. Every extra sentence in your system prompt, every additional document chunk retrieved by RAG, every turn of conversation history you include, all add to prefill time and push TTFT up.

Time Per Output Token (TPOT)

Once the first token appears, TPOT measures how fast subsequent tokens stream in. This is purely the decode phase.

A TPOT of 100 milliseconds per token would be 10 tokens per second per user, or roughly 450 words per minute, which is faster than a typical person can read. So 10 tokens/second is the rough threshold where streaming feels smooth rather than halting.

Below that, say 3-4 tokens/second, users start reading faster than the model writes. That friction is surprisingly noticeable and ruins the product's feel, even when the answers are good.

End-to-End Latency

The total time from the request submitted to the response being complete. Roughly: TTFT + (TPOT × number of output tokens).

For most product decisions, TTFT matters more than end-to-end latency, especially in streaming. If users can see tokens appearing, they’re engaged. A response that starts in 400ms and takes 8 seconds to complete feels very different from one that takes 4 seconds before anything shows up and completes in 5.

Streaming is not just a nice-to-have. It’s an inference architecture decision with real product impact. If your team hasn’t enabled streaming, it’s worth asking why.

Why Your Prompt Length Is an Infrastructure Decision

This is the thing I see PM teams miss constantly.

Every product decision that changes the size of your prompt is also an infrastructure and cost decision. Adding a more detailed system prompt: longer prefill, higher TTFT, and more cost. Retrieving more document chunks in RAG: same. Keeping longer conversation history: same.

This contrast is why commercial LLM APIs charge input tokens at a considerably lower rate than output tokens; prefill (processing input) is more compute-efficient per token than decode (generating output). But “cheaper per token” doesn’t mean “free.” At scale, a 500-token increase in average prompt length across a million daily requests adds up fast.

The practical question for PMs: do you know what your average prompt length is? Most teams don’t. Your tracing data will tell you (I covered how to get that visibility in my LLM Tracing 101 post). If you don’t have tracing, you’re flying blind on one of your biggest cost drivers. If you don’t have that number today, create a ticket this week to add prompt length to your tracing spans. It’s one data point that changes a lot of conversations.

What Batching Means for Your Users

When multiple users hit your product at the same time, the inference server doesn’t run each request separately. It batches them together, processing multiple requests on the GPU simultaneously to use the hardware more efficiently.

This is good for costs and throughput. But it creates a tradeoff.

The naive approach (static batching) waits until a batch is full before processing any of them. That means early arrivals in a batch wait for late arrivals before getting their TTFT. Not great for the users who showed up first.

Modern inference servers use continuous batching, with requests joining and leaving the batch dynamically as they complete. Any completed request is immediately removed from the batch, and the batch space is filled with the next request in line. This keeps TTFT lower across the board.

Why does this matter for PMs? Because when your team says “we need to optimize our inference server,” this is often what they’re talking about. And when you’re evaluating hosted inference providers (OpenAI, Anthropic, Together AI, Fireworks), their batching strategy directly affects your users’ experience during traffic spikes.

The Fundamental Tradeoff: Latency vs. Throughput

Here’s the one thing I want every AI PM to internalize from this post.

Latency and throughput pull in opposite directions.

Latency is about the speed of individual requests: how fast does a user get their answer?

Throughput is about the system capacity, how many users the system can serve per second.

Techniques that improve throughput (larger batches, more requests processed together) often increase latency for individual requests because each request waits longer before being processed. Techniques that minimize latency (smaller batches, dedicated resources) reduce how many concurrent users you can serve efficiently.

There’s no free lunch. When hosting and optimizing LLM inference, there’s always a balance between these two key goals.

The product question that determines where you land on this tradeoff: what does your user actually need?

A real-time customer support chatbot needs low TTFT above all else. A user who waits 4 seconds for the first word will lose trust fast. Throughput matters less; you’d rather serve 70 concurrent users well than 200 users with a 5-second TTFT.

A batch document summarization tool is the opposite. Nobody is watching tokens stream in. They submitted a job and will come back when it’s done. High throughput, higher latency tolerance.

A coding assistant sits somewhere in between. TPOT matters a lot (tokens need to stream faster than the developer can read), but TTFT can be a bit higher because developers expect a brief “thinking” pause.

Here’s a quick reference for how to think about SLO priorities by use case:

I teach my students to define their latency SLO (service level objective) before touching infrastructure. What TTFT is acceptable for your use case? What TPOT makes the experience feel broken? If you don’t have those numbers, every infrastructure conversation is just guessing.

Cold Starts: The Latency Spike Nobody Warns You About

One more thing worth knowing, especially if you’re deploying your own models rather than using hosted APIs.

When a model hasn’t been used recently, it may be unloaded from GPU memory to save costs. The next request that comes in has to wait for the model to load back onto the GPU before inference can even begin. This is called a cold start, and it can add anywhere from a few seconds to 30+ seconds of latency on that first request, longer if the model weights have to be loaded from disk in a self-hosted setup.

If your product has uneven traffic patterns, is busy during business hours, and quiet overnight, cold starts can create a painful experience for the first users each morning. The fix is to keep models “warm” by running occasional dummy requests, or to use an inference provider that handles warmup for you.

Most managed API providers (OpenAI, Anthropic, Google) handle this invisibly. If you’re self-hosting or using a serverless GPU provider, it’s your problem to solve.

What This Means for Product Decisions

A few concrete takeaways for your next sprint planning or infrastructure conversation:

On prompt design: Every token in your prompt costs money and adds latency. Audit your system prompt. I’ve seen teams with 4,000-token system prompts that could be cut to 800 without losing anything meaningful.

On streaming: If your product isn’t streaming responses, add it. It doesn’t change how fast the model runs; it changes how fast the experience feels. That’s a product decision, not just an engineering one.

On model selection: Smaller models have faster TPOT. GPT-4o mini and Claude Haiku are not just cheaper, they’re faster per token. For use cases where quality doesn’t require the largest model, the latency improvement is real and user-facing. I covered how to think about right-sizing model selection in my post, "Use the Smallest Capability That Works."

On SLOs: Define your latency targets before you build. TTFT under X milliseconds for Y% of requests. TPOT above Z tokens/second. Without those numbers, your engineers can’t make good tradeoffs and you can’t evaluate whether your infrastructure is working.

Inference is the part of the AI stack that’s invisible until it breaks and then it’s the only thing your users can talk about. Understanding the two phases, the three metrics, and the latency-throughput tradeoff gives you enough foundation to have informed conversations with your engineering team about the decisions that directly affect your users.

You don’t need to know how to configure vLLM or tune a KV cache. But you do need to know why your product felt slow that Tuesday afternoon when traffic spiked, and what questions to ask to figure it out.

That conversation starts with knowing what inference actually is.

What’s the inference-related issue that’s bitten your product hardest, cold starts, TTFT spikes during traffic peaks, something else? Drop it in the comments. And if this post helped clarify something your team has been debating, share it with an AI PM or engineer who’d find it useful. This is the kind of thing that’s easier to understand together.

If you’re not subscribed yet, I publish posts like this every week at aipmguru.substack.com, practical AI PM concepts, no hype.

Subscribe now

Related reading:

• Understanding Context Windows: A Critical Concept for AI PMs

• LLM Tracing 101: What It Is, Why It Matters, and When You Actually Need It

• The Hybrid Chatbot Stack: When to Cache, Route, or Generate

That one sentence is the difference between a candidate who will build responsible AI products and one who will ship hallucination factories and wonder why users don’t come back.

I know this question is coming in my AI PM interviews. Every prep guide flags it. Every hiring manager roundup mentions it. And more than almost any other question, it exposes whether you’ve thought carefully about what these models actually do or whether you’ve been treating them like magic.

So I went deep on it. Here’s where I landed.

What You’ll Learn (6-minute read)

• Why the “it’s a bug” framing is the first mistake candidates make

• Why hallucinations happen at a technical level (and why that framing matters)

• A three-layer approach to addressing them in products

• How to answer the follow-up question that most candidates stumble on

• What separates a good answer from a great one

The Framing Mistake That Kills Most Answers

Most candidates approach this question as a mitigation exercise. They jump straight to RAG, guardrails, output validation, and lower temperature. All real tools. But leading with them tells the interviewer something that you think hallucination is a problem to be patched, not a property to be designed around.

The difference matters more than it sounds.

If hallucination is a bug, you’re waiting for the next model release to fix it. If it’s a property, you’re designing every product decision around it, the UX, the trust signals, the error states, the domain scope. Two completely different product philosophies. Interviewers can usually tell which one you hold within the first 90 seconds.

I actually tried the mitigation-first approach in a mock session with Claude early in my prep. Led with RAG. My interviewer (Claude) asked: “But what if the retrieved document is wrong?” I didn’t have a clean answer. That’s when I realized I was treating this as a technical checklist problem instead of a product design problem.

Why Hallucinations Actually Happen

I covered the full mechanics in my post on how LLMs actually work, but the short version relevant here is: LLMs predict the most likely next tokens based on patterns in the training data. They’re optimizing for plausibility, not factual accuracy.

There’s no internal fact-checker. The model can’t distinguish between “I know this” and “this sounds right.”

When it hits a question without strong pattern matches, it generates confident-sounding text that follows the expected format, even if the underlying content is completely invented. That’s what “confidently wrong” means at the architectural level. Not a glitch. Not a failure state. The natural output of a system that predicts likely continuations rather than verifying facts.

You’re not working with a system that occasionally makes mistakes. You’re working with a system that fundamentally cannot verify its own outputs against reality. That’s the frame worth opening with in an interview.

Three Layers Worth Walking Through

Read more

“How will you know if this feature is working?”

The answers I get fall into one of three patterns. Some students jump straight to model metrics: accuracy, latency, and F1. Some go straight to product metrics: adoption, engagement, CSAT. A few go straight to business metrics: cost reduction, revenue impact.

All three are right. And all three are incomplete on their own.

The reason I start there is that the mistake I see most often in AI PRDs isn’t a missing metric. It’s a missing layer. Teams pick one frame and treat it as the full picture. Then they ship something that looks great by one measure and quietly fails by another.

What I teach and what I use in my own PRD work is a three-layer stack. Business metrics, product metrics, model metrics. In that order. Each layer answers a different question. Each one is incomplete without the other two.

What You’ll Learn (7-minute read):

• Why AI features require metrics at three distinct layers

• How to define each layer in a PRD, with concrete thresholds (not hand-wavy goals)

• A full worked example you can adapt directly

• The one-layer trap and how to spot it in your own drafts

Start With the Business Layer

I always tell my students: before you write a single acceptance criterion, write the business metric.

Not because it’s the most interesting layer. Because it’s the one that answers “why are we doing this at all?” If you can’t answer that with a specific, measurable outcome, you don’t have a PRD. You have a feature request dressed up in a template.

Business metrics for AI features usually live in one of four places: revenue lift (upsell attach, expansion, pricing power), cost reduction (support deflection, manual hours replaced), quality and retention (NPS, churn, CSAT on the specific workflow you’re improving), or risk reduction (compliance outcomes, error rates in high-stakes processes).

For a support summarization feature, a business metric might be: reduce average handle time per ticket by 20% within 6 months, while keeping CSAT at or above 4.5/5.

Notice there’s a number and a timeframe. “Improve efficiency” is not a business metric. That’s a wish.

One test I find useful: if this business outcome could be achieved just as well without AI, say so in the PRD. Better workflow tooling, better training, better templates, these are real alternatives. Being explicit about why AI is the right approach for this outcome forces a healthier conversation before a single line of code gets written.

Then the Product Layer

Once the business “why” is clear, the product metrics answer: how will user behavior tell me we’re on track before the business numbers move?

Business metrics are lagging. They tell you what happened. Product metrics are leading. They tell you whether you’re headed in the right direction.

This is where PM instincts matter most. A very accurate, very fast model can still produce a feature nobody trusts. Product metrics expose that gap before it becomes a customer problem.

The signals I reach for:

Adoption and depth. What percentage of eligible users try the feature? Among those who try it, how often do they come back? For AI features, depth matters more than initial activation. The first experience is often rough. Users who return after that first rough experience are telling you something real.

Trust signals. Thumbs up/down on outputs, feature-level CSAT, and qualitative feedback volume. Trust is a leading indicator for AI features in a way that it isn’t for traditional ones. If users don’t trust the output, they’ll stop engaging long before your business metrics show it.

Behavioral shift. Did the AI change how users work, or did they use it once and default back to the old way? For support summarization: are agents actually sending tickets faster, or reading the summary and rewriting it from scratch? Those two behaviors look identical in a usage report. They mean completely different things.

For the same support feature: within 3 months of GA, 60% of active agents use AI summaries on at least 30% of tickets per week. Agent satisfaction at 80%+ “useful” votes. Behavioral shift is tracked by whether the summary-to-send workflow replaces the previous manual flow, not just whether summaries are being generated.

Then the Model Layer

Model metrics come last in the writing order. Not because they matter least — they don’t. Because they should be derived from the product and business you’ve already defined, not chosen in isolation.

This is something I push hard on in class. If you define model quality thresholds first, you end up optimizing for the wrong thing. “Let’s get to 95% accuracy” is a meaningful goal only if 95% ’s what the product actually needs to earn user trust and hit the business target. Without layers one and two, that number is just a guess.

Quality. For classification: accuracy, precision/recall, F1. For generative outputs: human evaluation win rate. For summarization specifically: “What percentage of AI summaries do human evaluators rate as accurate enough to send without edits?” That’s not a model question. It’s a product question answered with a model threshold.

Reliability and safety. Hallucination rate, policy violation rate, and PII exposure rate. These aren’t compliance checkboxes. They’re the floor below which user trust collapses. Define them explicitly.

Latency. P95 response time is a UX metric that lives in the model layer. Users won’t wait. How long is too long for this specific workflow? That number should come from your product metric targets, not from what the model team thinks is achievable.

For support summarization: 90% of summaries rated “accurate enough to send without edits” in human eval. Fewer than 1% expose PII not already visible in the original ticket. P95 latency under 3 seconds from ticket submit to summary visible.

Those thresholds aren’t arbitrary. They’re derived from what agents need to trust the feature and what the business needs to see handle time actually drop.

A Full PRD Example

Here’s how I’d write the success section for “AI Draft Replies for Support Tickets.”

Business metrics

Reduce support cost per resolved ticket by 15% within 9 months. Maintain CSAT at 4.6/5 or above across all support interactions.

Product metrics

Within 3 months of GA, 70% of agents use AI draft replies on at least 40% of eligible tickets per week. For tickets where AI drafts are shown, agents send a final reply 30% faster than the pre-feature baseline. Agent feedback at 80%+ “draft was helpful” votes.

Model metrics

Quality: 85%+ of drafts rated “send with minor or no edits” in blind human evals. Safety: PII leak and policy-violation rate at 0.5% or lower on sampled interactions, with automated flags above threshold. Latency: P95 draft generation under 2.5 seconds.

The causal chain here is intentional. If the model hits its quality and latency targets, agents should find drafts useful. If agents find drafts useful, we should see faster replies and higher adoption. If replies are faster and quality holds, cost per ticket drops without hurting CSAT.

That chain is what you’re writing down in the PRD. Not three separate lists of metrics. The theory of how they connect.

The One-Layer Trap

The pattern I correct most often, both in student PRDs and in the mock interview feedback I give, is what I call the one-layer trap.

It looks like this: the PRD has model metrics (accuracy, latency, test set performance) and maybe product metrics (adoption, engagement), but no business success criteria with actual thresholds. Or it has strong business and product goals but no defined quality bar for the model, which means “good enough” gets decided by whoever is loudest in the launch review meeting.

The fix is the same in both cases. Ask the two bridging questions before you finalize any layer:

If the model hits these numbers, what will users do differently?

If users do that thing differently, what company-level metric moves and by how much?

If you can’t answer both with specifics, the PRD has a gap. Find it before engineering does.

When I write the success section of an AI PRD, I don’t move on until I’ve answered three questions cleanly:

If this works, how does the company notice? Business metrics with specific thresholds and timeframes.

How will user behavior tell me it’s working before the business numbers move? Product metrics with adoption, depth, and trust signals.

What does “good enough” mean for the model, defined by what the product and business actually need? Model metrics derived from layers one and two, not chosen in isolation.

That’s the stack. Three layers, one causal chain, written down before you ship.

For a deeper look at how model evaluation fits into this, I wrote about building your first AI eval step by step in "How to Build Your First AI Eval: A Step-by-Step Walkthrough Using the Anthropic Console."

What layer do you find hardest to define in practice? Business metrics that are specific enough, or model thresholds that are actually derived from product needs? I’d love to hear what’s coming up in your PRDs.

I told them I’d build a real AI workflow, one that scrapes AI news, writes a podcast script, and generates audio. Live demo. The whole thing.

I said yes confidently. I had two months. I build things. I know AI. I could figure out n8n.

Except for three weeks before the event, I still hadn’t opened it.

Not because I wasn’t building. I was - with Bolt, Lovable, Claude Code. I’m comfortable in those environments. But n8n felt different. The nodes, the wires, the HTTP requests. Every time I looked at a screenshot of someone’s workflow, my brain went quiet in a way it doesn’t usually go quiet around technical things. And that quiet was starting to feel like a message.

I genuinely considered pivoting to a Claude Code demo. Something I already knew. Something I could walk through confidently without the risk of freezing in front of a room.

One week out, I decided that wasn’t acceptable.

If I don’t try, I will always live in fear of this.

So the Thursday before the event — six days out — I cancelled my meetings and started.

Day 1: Two Hours. Six Nodes. One Audio File.

I opened Claude and started talking. Not coding. Just describing what I wanted: pull AI news from RSS feeds, have Claude prioritize the best stories, write a podcast script, send it to ElevenLabs, get audio back.

Claude walked me through it, piece by piece. And I want to be clear, Claude could have generated the entire workflow on its own. But I wanted to understand how each node worked, how the pieces connected to each other. So we went slow. One node at a time, me asking questions the whole way.

Two hours later. TWO whole hours. Six nodes.

Messy. But functional. And at the end of that session, I had an audio file.

An actual podcast episode. Generated from news I hadn’t touched.

I sat there for a minute just staring at it.

I’ve been building with AI for a while now — Bolt, Lovable, Claude Code. I’m not new to this. But n8n was the tool I’d been circling, the one I kept telling myself I’d get to eventually. And right there, staring at that audio file, something clicked. Not a new skill. A missing piece. The workflow layer I’d been avoiding finally made sense in a way that slides and frameworks never quite captured.

That’s the difference between teaching a thing and doing the thing. I know that difference. I talk about it constantly. And I still had to learn it again.

Three Days Later: 14 Nodes

By the time I got to the event, the workflow had grown.

Four RSS feeds: ArXiv ML, TechCrunch, Hacker News, and MIT Technology Review. A merge node pulls everything together. Claude is summarizing and prioritizing per my instructions. Claude is writing the script. A human approval gate in the middle, because I wanted eyes on the script before audio got generated. ElevenLabs for the audio. Google Drive is storing the MP3 and the transcript. Notion is getting updated with the episode record.

Fourteen nodes. Three days.

I wasn’t starting from zero, knowing how to think about AI architecture, how to break a workflow into steps, how to write a good prompt, all of that mattered. The n8n-specific learning was maybe 30% of the work. The other 70% was the builder thinking I already had.

That’s worth saying, because I think a lot of PMs underestimate how much they already know. The tool is learnable. The thinking is the harder part, and if you’ve been doing the thinking, you’re closer than you think.

The Demo Day

Here’s what actually happened before I got to the demo.

I had a tech glitch that made me quit Zoom entirely and rejoin. Right before I was supposed to start. In front of 45+ people.

I got back in and said: “Sometimes AI can only get us so far.” Got a laugh. Kept going.

And then I built the 6-node version live, in front of the audience, in about 20 minutes. No pre-built shortcuts. Just the workflow, one node at a time, exactly the way I’d learned it six days earlier.

It worked.

But here’s the moment I actually want to talk about. During the build, I walked the audience through the human approval gate — the node that pauses the workflow and shows me the script before audio gets generated. I explained why it’s there.

A few days earlier, while I was testing, the workflow had produced a script that opened with: “Hello, this is the news from March 24, 2024.”

Two years off. Completely confident. Completely wrong.

That’s why the approval gate exists. Not because I don’t trust Claude. Because I know that every AI system produces wrong outputs sometimes, and the question isn’t whether it will happen, it’s whether you’ve designed for it. The approval gate is me staying in the loop on the one step where a mistake would actually matter.

I told that story live. And I think it landed harder than any of the framework slides I showed.

The audience got quiet, the way audiences do when something real just happened.

“I Finally Started Calling Myself an AI Builder.”

I said that in my intro. Out loud. To 45 strangers.

It felt weird to say. I’ve been an AI PM since 2019. I’ve worked at Disney, Nike, and Amazon. I teach AI Product Management at the University of Washington. But “AI builder” felt like a different claim, one I wasn’t sure I’d earned yet.

Building this workflow is what got me there. Not because n8n is hard (it isn’t, once you start). But because I stopped waiting until I felt ready and just built the thing. That’s what builders do.

Three Tools I Built for the Talk

While preparing, I realized the demo wasn’t enough. I needed thinking tools. Things that would help people make better decisions before they build, not just watch a workflow run and go home inspired.

So I built three of them. A pressure-tester for build decisions. A 9-role PRD critic. And a 35-prompt library organized by situation.

They’re available below for paid subscribers, along with the workshop starter template and the simplified 6-node build guide you can use to build your first workflow this afternoon.

The AI PM Thinking Partner is a Claude skill that acts like a senior AI PM who will push back before you commit to building something. It asks questions first — because that’s what good mentors do before giving opinions. Then it names the pattern it’s seeing. Then it gives you a verdict: BUILD IT, VALIDATE FIRST, SIMPLIFY FIRST, or DON’T BUILD THIS YET.

[Download the AI PM Thinking Partner →]

The PRD Review Critic reads your product brief and interrogates it from nine stakeholder perspectives at once — Legal, Engineering, Design, AI Research, and more. The output is a traffic-light scorecard with specific, uncomfortable questions under each Red and Yellow item. Not “have you considered privacy?” — the version with actual teeth.

[Download the PRD Review Critic →]

The AI PM Prompt Library is 35 copy-paste prompts organized by situation: scoping a build, writing or fixing a prompt, evaluating output, explaining AI to stakeholders, prepping for interviews. Each prompt includes a note on why it’s designed the way it is, so when your situation doesn’t fit exactly, you know what to adapt.

[Download the AI PM Prompt Library →]

The workshop starter template is the n8n workflow from the live demo — importable, ready to wire up with your own API keys.

[Download the workshop starter template →]

Most product managers think they’re good with AI because they use ChatGPT to clean up emails or summarize docs. That’s useful. I’m not dismissing it.

But there’s another level. Where you’re not just using AI tools, you’re building with them. Workflows that run while you do the work you’re actually supposed to be doing.

The shift to get there isn’t really about the technology. I knew enough to build this in three days because I’d been doing the thinking for a long time. What I’d been missing was the willingness to sit with one specific discomfort long enough to get through it.

I cancelled my meetings on a Thursday because I was tired of being afraid of something I hadn’t tried. Six days later, I demoed a 14-node workflow to a room full of product managers.

You probably know more than you think. You just have to start.

What’s the thing you’ve been circling around, telling yourself you’ll get to eventually? Drop it in the comments.

Without tracing, you can’t answer that question.

You can see the bad output. You can see the user’s input. But everything in between, the six steps the system took before generating that response, is a black box. You’re debugging a pipeline by staring at its endpoints and guessing what went wrong in the middle.

That’s what tracing fixes. And for AI PMs shipping production products, I think it’s one of the most underappreciated tools in the stack.

What you’ll learn (8 minutes):

• Why evals aren’t enough for production

• What a trace actually looks like, span by span

• Where hallucinations really originate in the seven context engineering components

• Which tool to start with

• When tracing is overkill.

Evals Tell You It Can Work. Tracing Tells You It Is Working.

If you’ve built evals for your LLM product (and I wrote about how to get started with that here), you already know how to test whether your system gives good answers to known questions. That’s necessary. But it’s not sufficient.

Here’s the gap. Evals test a curated set of inputs against expected outputs. Tracing records what actually happens when real users, with messy, unpredictable queries, hit your system in production.

Traditional software crashes when it fails. You get a 500 error, a stack trace, and a Sentry alert. LLM products fail silently. The model returns a confident, grammatically perfect, completely wrong answer. No error code. No crash. Just a user who quietly loses trust in your product.

I think this is the single biggest operational difference between traditional and AI products. The failure mode is silence. And you can’t fix what you can’t see.

So What Is Tracing, Exactly?

Think of it like a flight recorder for your AI pipeline. Every time a user sends a query, the trace captures a complete, timestamped record of every step the system took to produce the response.

In traditional distributed systems, a “trace” follows a request as it bounces between microservices. Same concept here, but the hops are different. In an LLM application, a trace follows a user query through prompt construction, retrieval (if you’re using RAG), the model call itself, any tool use or function calls, post-processing, guardrail checks, and the final response.

The key distinction from logging: logs tell you that something happened. Traces tell you why it happened in that order, with that result, at that cost.

A log might say: “LLM call completed in 1,230ms.”

A trace shows: the user asked about refund policies, the system retrieved 4 document chunks (with their similarity scores), assembled a 2,847-token prompt including the system instructions and conversation history, called GPT-4o at temperature 0.3, generated 340 tokens in 1,230ms, passed the guardrail check, and returned the response. Total cost: $0.004.

That level of visibility is what lets you answer the hallucination question. Was the retrieval bad? Did it pull the wrong documents? Was the prompt too long, and did the model lose focus? Did the system instructions conflict with the retrieved context? You can see exactly where things went sideways.

The Anatomy of a Trace

Traces have a hierarchy that takes about five minutes to understand. Here’s the vocabulary:

Here’s what a trace actually looks like for a RAG-based chatbot. A user asks: “What’s the refund policy for international orders?”

Span 1: Query Processing (45ms)

The raw user input gets cleaned up and prepared. Maybe you’re extracting intent or checking for prompt injection attempts.

Span 2: Embedding Generation (120ms)

The query is passed to your embedding model (say, text-embedding-3-small) and converted into a vector. If your embedding API is slow or rate-limited, this span tells you.

Span 3: Vector Search (85ms)

That vector goes to your database for similarity search. Four document chunks come back, each with a relevance score. This is where most RAG problems actually hide. Bad chunking, low relevance scores, wrong top-k values. The retrieval span exposes it all.

Span 4: Prompt Assembly (15ms)

The system instructions, retrieved chunks, conversation history, and user query get assembled into the final prompt. 2,847 tokens total. This span shows you exactly what the model is seeing, which matters because if the context is contradictory or too long, you’ll spot it here.

Span 5: LLM Generation (1,230ms)

The actual model call. This is typically the longest and most expensive span. You can see the model used, temperature setting, token counts (input and output), and the raw response.

Span 6: Guardrail Check (35ms)

Output validation. Did the response pass your safety filters? Was it grounded in the retrieved context?

Total: 1,530ms. Cost: $0.004.

And now you can see exactly where time and money went. If latency spikes, you know which span is responsible. If quality drops, you can trace back through the pipeline and find the step that broke.

So, Where Do Hallucinations Actually Start?

This is the question from the opening. The honest answer: it depends. Hallucinations don’t have one origin point. They can creep in at almost every stage. But tracing lets you narrow it down.

Let me map it back to the seven context engineering components from that diagram:

Component 1: System Instructions → If your system prompt is vague or contradictory, the model has more room to improvise. You can’t trace inside the model’s reasoning, but you can inspect exactly what instructions were sent. Tracing catches this at Span 4 (prompt assembly), where you’ll see the full prompt, including system instructions.

Component 2: Background Context → Stale or inaccurate domain knowledge baked into the prompt. Traceable at Span 4. If your background context says the return policy is 30 days, but it changed to 14 days last month, the trace shows exactly what context was injected.

Component 3: Conversation History → In multi-turn conversations, if the history is too long and gets truncated, the model may lose important earlier context. Traceable at Span 4 by inspecting how much history was included and where it was cut.

Component 4: Retrieved Knowledge (RAG) → This is the big one. Bad retrieval is probably the most common source of hallucinations in RAG systems. The retrieval span (Span 3) shows you the similarity scores, the documents returned, and whether they were actually relevant to the query. If the model hallucinated because it got irrelevant chunks, you’ll see it here.

Component 5: Available Tools → If your system calls external tools or APIs and receives bad data, the model may incorporate that into its response. Traceable as a tool spans the nested trace.

Component 6: Output Format → Less of a hallucination source, more of a formatting issue. But if the output format constraints are confusing, the model might generate structurally weird responses. Visible in the prompt assembly span.

Component 7: User Query → Ambiguous or adversarial queries can trigger hallucinations. The trace starts with the raw input, so you can see exactly what the user asked and whether your preprocessing changed it.

And then there’s the one source tracing can’t catch: training data. If the model learned incorrect associations during pre-training, those associations are baked in before your pipeline even runs. No amount of tracing will surface that. This is where evals matter, because you can test for known factual errors even if you can’t trace their origin.

Most hallucinations you’ll deal with in production come from Components 1 through 4. System instructions, context, history, and retrieval. And all of those show up in traces. That’s the point. Evals catch what the model gets wrong. Tracing shows you why it got it wrong. Different jobs, both necessary.

How This Differs from Traditional APM (Application Performance Monitoring)

If you’ve worked with tools like Datadog or New Relic for regular software, LLM tracing looks familiar at first glance. But it tracks fundamentally different things.

Traditional APM monitors latency, error rates, and throughput. LLM tracing adds token counts (input and output), model parameters (temperature, top_p), retrieval quality scores, prompt versions, cost per request, and the actual content of inputs and outputs.

That content piece is what makes this a category entirely different. You’re not just monitoring whether the system responded. You’re monitoring whether the response was good. Traditional APM can tell you “the API returned 200 OK in 450ms.” LLM tracing can tell you “the API returned a confident answer that contradicted the retrieved documents.”

There’s a privacy consideration worth flagging early. Because traces capture full prompts and responses, you’re suddenly storing user inputs and model outputs. That has implications for compliance, data retention, and PII handling that traditional APM doesn’t have to worry about. A reasonable starting point: default to not logging raw prompts and responses in production. Log token counts, model names, latency, and trace IDs. Turn on full content capture selectively when you’re debugging a specific issue.

What Should a PM Actually Look At?

You don’t need to read raw trace data. (That’s your engineering team’s job.) You need dashboards that answer PM questions.

Here are the ones I think matter most:

“What’s our average cost per conversation?” Not per API call. Per conversation. This is your unit economics question, and it comes directly from aggregating trace-level cost data across sessions. If you’re building pricing models or estimating margins, this number matters more than anything on your model provider’s invoice.

“Which queries are going to the LLM when they shouldn’t be?” If you’ve built a hybrid stack with caching, rules-based routing, or tiered models (I covered this architecture in my hybrid chatbot post), traces show you whether the routing is working. Maybe 40% of your queries are hitting your expensive model when they could have been handled by a cheaper one or a cached response.

“Where are users dropping off mid-conversation?” If users ask one question and never come back, or abandon a multi-turn conversation at step 3, traces correlated with session data tell you where. Was the response too slow? Too generic? Did the model lose context?

“How did last week’s prompt update affect quality?” This connects tracing back to evals. Your best eval test cases don’t come from synthetic examples you imagined at your desk. They come from real production queries that failed. Traces give you those failing examples, and you feed them back into your eval suite. That’s the iteration loop.

When You Need It (And When You Don’t)

Not every AI product needs full tracing from day one. I think about this as a maturity question.

Just prototyping? Console testing and manual spot-checks are fine. Don’t over-invest in infrastructure you’ll probably rebuild anyway.

I’ll be honest: the first RAG pipeline I built for a course demo had zero instrumentation. When a student asked why it kept hallucinating product specs, I had to shrug and say, “Let me look at the prompts.” We spent 20 minutes manually reconstructing what the retrieval had probably returned. That’s when I started treating tracing as a day-one requirement, not a “we’ll add it later” item.

Internal tool with a small user base? Basic logging (inputs, outputs, timestamps) might be enough. You can probably debug issues by just looking at recent queries.

Production app with real users? You need tracing. Silent failures at scale are not debuggable with logs alone.

Multi-step agent or RAG pipeline? You really need tracing. When you have 6+ steps in a pipeline, the combinatorial space of “what went wrong” gets unmanageable without span-level visibility.

Regulated industry (healthcare, finance, legal)? Tracing isn’t optional. It’s your audit trail. You need to show exactly what the system did with each request and why.

The cost-complexity tradeoff is real. Most tracing tools charge per trace or per span. At scale (millions of requests), this becomes a meaningful line item. Some teams start by tracing 100% of requests in development, then sampling 10-20% in production, with full tracing triggered automatically for any request flagged by guardrails or receiving negative user feedback. That’s a reasonable starting point.

Where to Start with Tools

I’m not going to do a full comparison (the space moves fast). But if a student asked me which one to try first, here’s where I’d point them:

If you want open-source and self-hostable: Start with Langfuse. It’s built on OpenTelemetry, which means you’re not locked into their ecosystem. You can self-host if you have data residency requirements, and the community is active.

If your team is already building on LangChain: LangSmith has the tightest integration. The onboarding is fast because it understands your chain structure natively.

If you’re already a Datadog shop: Datadog LLM Observability is the path of least resistance. Your LLM traces appear alongside your existing APM data, so you can correlate model latency with database queries or infrastructure issues without switching tools. Worth noting: LLM Observability requires an APM-enabled Datadog plan, so check your tier before assuming it’s included.

What’s Next

If you’ve already built evals, tracing is the natural next step. Evals confirm your system can do the job. Tracing confirms it is doing the job, request by request, in production.

Start simple. Instrument one pipeline. Look at 50 traces. I think you’ll immediately see patterns your evals missed: queries you never thought to test, retrieval failures you didn’t anticipate, cost hotspots you didn’t know existed.

And the next time someone asks, “Where did the hallucination come from?”, you’ll be able to pull up the trace, point to the span, and say, “Right here. The retrieval returned irrelevant documents at step 3. That’s what the model was working with.”

That’s a very different conversation than shrugging and saying “LLMs sometimes make things up.”

If you’ve shipped an LLM product, what’s the first thing tracing showed you that you couldn’t see before? Retrieval failures? Cost surprises? Users asking things you never anticipated? I’m curious what the first trace actually reveals in practice.