If you enjoy AI Leadership Edge, it would mean the world to me if you invited friends to subscribe and read with us. If you refer friends, you will receive benefits that give you special access to AI Leadership Edge.

How to participate

1. Share AI Leadership Edge. When you use the referral link below, or the “Share” button on any post, you'll get credit for any new subscribers. Simply send the link in a text, email, or share it on social media with friends.

Refer a friend

2. Earn benefits. When more friends use your referral link to subscribe (free or paid), you’ll receive special benefits.

• Get a 1 month comp for 3 referrals

• Get a 3 month comp for 5 referrals

• Get a 6 month comp for 25 referrals

Visit the leaderboard

To learn more, check out Substack’s FAQ.

Thank you for helping get the word out about AI Leadership Edge!

One documented enterprise deployment ran its proof of concept for about $1,500 a month in API usage. The results looked strong. Leadership approved full production, and the monthly bill at real-world volume landed just over $1 million. That is a 700X jump from pilot to production, and no business case survives a multiplier like that.

The case comes from an analysis of enterprise LLM deployments, and while the number represents a worst case, the mechanics behind it are ordinary. At pilot scale, a cost of $0.10 to $0.50 per agent request is easy to absorb and even easier to present as a savings story. At 10,000 users, the same request rate produces a monthly infrastructure bill that makes the original spreadsheet unrecognizable. Most teams run those numbers after the architecture decision is locked, which happens to be the most expensive possible time to learn them.

The Multiplier Hiding Inside Every Agent Request

A chatbot query triggers one inference call. An AI agent working through a task plans, calls tools, evaluates results, and loops back when something fails. Gartner’s analysis from earlier this year found that agentic workflows consume between 5 and 30 times more tokens per task than a standard chatbot, with a single user request often triggering 10 to 20 separate model calls behind the scenes.

Four mechanics drive that multiplier.

Reasoning loops sit at the center. Every pass through plan, act, and evaluate fires at least one model call, and complex tasks can take dozens of passes before the agent settles on an answer.

Context accumulates. Agents carry system prompts, tool definitions, and step history into every subsequent call. All of it gets re-sent each time, so the token cost of step twelve includes the freight of steps one through eleven.

Tool calls stack their own costs on top. Web searches, database queries, and code execution each add latency and expense beyond the model call that triggered them.

Retries compound everything above. When a tool returns an unexpected schema or an output fails validation, the agent tries again. Each retry is a fresh trip through the loop, and you pay for the attempt whether the task succeeds or fails.

One documented example makes the point better than any abstraction. A coding agent assigned to fix a one-character typo in a README consumed over 21,000 input tokens listing issues, branching, committing, and opening a pull request. A trivial fix, wrapped in an expensive workflow.

Cheaper Tokens, Bigger Bills

Per-token pricing has collapsed. Inference for a GPT-3.5-level model fell from $20 per million tokens in late 2022 to $0.07 by October 2024, roughly a 280x drop in two years, and Gartner projects inference on trillion-parameter models will cost 90 percent less by 2030. Enterprise AI bills keep rising anyway, because total token consumption is growing faster than prices are falling. More capable agents run more reasoning loops, call more tools, and burn more tokens per completed task. Capability and cost move together by design, since quality in these systems comes from iteration rather than single-pass generation.

Uber ran into the same problem at scale. The company rolled out agentic coding tools to roughly 5,000 engineers, and heavy users racked up between $500 and $2,000 per month each, burning through the annual AI budget in about four months. The pilot had only ever tested one engineer, and nobody had modeled what concurrency at that scale would cost.

Run the Production Math Before the Architecture Locks

The forecast that prevents all of this takes about an afternoon to build. Start with cost per completed task from your pilot data, repriced at full production rates rather than free-tier or discounted credits. Multiply by the ratio of production users to pilot users. Then apply a burstiness factor of 3 to 5x, because production traffic spikes and runs in parallel in ways a pilot never exercises. If the resulting number breaks the business case, a spreadsheet is a far cheaper place to find out than an invoice.

Cost per successful task is the metric worth anchoring on. Cost per prompt and cost per session both hide failure. An agent that completes tasks cheaply but fails half the time and requires human cleanup costs far more than its dashboard suggests, and that gap stays invisible until you measure completion rather than activity.

In my work with client teams, the forecast conversation almost never happens at this stage. The pilot generates momentum, the demo impresses the steering committee, and the architecture gets approved on pilot economics. Everything downstream inherits that assumption.

Decide Which Steps Actually Need an Agentic Loop

Autonomous reasoning is the most expensive pattern in the stack, and most workflows only need it in a few places. Research on enterprise deployments suggests small language models can handle 60 to 80 percent of agent tasks at 10 to 30 times lower inference cost, with frontier models reserved for the steps that require genuinely complex reasoning.

A routing layer that classifies each step and sends classification, formatting, and retrieval work to smaller models can cut costs by 60 percent or more without touching quality where it matters. Caching does similar work on the input side. If the agent starts every task with the same system prompt and knowledge base, prompt caching can reduce input costs by roughly 90 percent. Retry caps close the remaining leak. Classify errors so some warrant a retry, some escalate to a human, and some fail gracefully before the loop becomes a line item.

The work is unglamorous. Walk the workflow step by step and decide where iteration earns its cost.

Build Cost Governance Before the Quarterly Surprise

Gartner predicts that more than 40 percent of agentic AI projects will be cancelled by the end of 2027, with escalating costs and unreliable outputs as leading causes. Most of those cancellations will trace back to the same sequence. The pilot succeeds, the deployment scales, the bill arrives, and the cost conversation happens under pressure with a CFO reading line items aloud.

Governance moves that conversation earlier, where it costs almost nothing. Track spend per task, per step, and per tool call, since aggregate metrics hide the one workflow that’s eating the budget. Set alerts on cost per successful task so anomalies surface in days instead of at quarter close. And assign an owner, because a cost that’s technically everyone’s job doesn’t get caught by anyone.

An agent can query the right table and return the right number for every account except one, because the exception on that account lives in an analyst’s memory, and no table records it.

Teams debugging this kind of failure usually start with the model. They tune prompts or switch providers, and the error rate barely moves, because the model was never missing intelligence. It was missing the fact.

In my work as a fractional CTO, this is the most common root cause I find when enterprise agents produce wrong outputs in production. The information required to do the job correctly exists. It exists in undocumented form, distributed across the memories and judgment calls of the people who have been doing the work.

Where the Knowledge Actually Lives

Every company runs on a layer of knowledge no system captures. The business rule each new hire learns from their manager in week three. The understanding that two product names in two different databases refer to the same SKU, reconciled by whoever pulls the report. The approval that officially requires three signatures but in practice needs one phone call.

Humans handle this layer so fluidly that most organizations forget it exists. The analyst who knows the pricing exception applies it without thinking. An agent querying the same systems has no idea the exception exists and no way to discover it, because the reconciliation happens in someone’s head.

Four Context Failure Modes

Across the agent deployments I’ve audited, four failure modes account for most wrong outputs in production.

Definitions stored as institutional memory

Finance counts a customer as active if they’ve paid in the last 90 days. Sales counts anyone with an open opportunity. Both definitions are correct inside their own department, and the difference lives in a shared understanding that never made it into a schema. An agent asked for the churn rate picks one of them and produces a number half the company will dispute.

Entity identity that resolves differently across systems

The ERP calls it PRO-ENT-2. The billing platform calls it Enterprise Pro v2. Everyone who works with both systems knows they’re the same product, and no table anywhere records that fact. Agents join on what the data says, so revenue for one product splits into two.

Process knowledge that was never written down

The documented workflow says contracts route through legal. The actual workflow routes anything under $50K through a template the paralegal maintains, a shortcut established years ago and passed along verbally. An agent following the documented process produces work that is technically correct and operationally useless.

Outputs that cannot be traced to a source

When a senior analyst approves an exception, the reasoning lives in her judgment. Nothing records why. When an agent later makes a similar call, nobody can audit the chain, because the precedent it needed was never a record. Provenance breaks the moment a decision depends on knowledge with no source of truth.

The Gap Between Having Data and Using It

The industry numbers describe the same distance. DataHub’s State of Context Management Report 2026, which surveyed 250 IT and data leaders, found that 88% claim to have fully operational context platforms, while 61% frequently delay AI initiatives due to a lack of trusted data.

Read those two numbers together and the picture sharpens. Nearly nine in ten organizations believe they have the infrastructure. Six in ten keep delaying launches on top of it. The distance between “we have the data” and “AI can use the data” is where most enterprise AI budgets currently leak, and it rarely shows up as a line item because nobody owns it.

A Different Kind of Investment

Investors have started pricing this gap. Tribal, founded by Salesforce, Wix, and Spot.io veterans, raised a $10 million seed round in May to build agents on a metadata fabric that maps a system of record’s business rules and dependencies before any agent acts. The founding team’s read of the market matches what I see inside companies. Building AI prototypes is easy. Shipping trusted change inside a live enterprise system is brutally hard, and the difficulty concentrates in the context layer underneath the model.

Most AI roadmaps are built to buy tools and integrate APIs. Closing the context gap looks different. It means interviewing the analyst who carries the pricing exceptions and encoding what she knows. It means building the entity resolution table that finally reconciles PRO-ENT-2 with Enterprise Pro v2. Someone has to write down the actual approval workflow, shortcuts included, and decide which ones the agent is allowed to follow. This is slow, unglamorous work that involves calendars more than compute.

Start With What Would Break

A practical first step I give clients requires no vendor. For each workflow an agent will touch, ask what would break if the most tenured person on that team left tomorrow. The answers form a map of your undocumented knowledge, and that map predicts where your agents will fail before you deploy them.

Then fund the extraction work explicitly. Give it an owner and a deadline, the same way you would any infrastructure project, because that is what it is. Encode the definitions, resolve the entities, and record the reasoning behind exceptions so outputs can be traced.

Your org chart is part of your data architecture. Every retirement and every reorg deletes records no backup will recover. Agents make that loss visible because they fail where humans compensated. The undocumented layer was always a liability. Agents just turned it into an error rate you can measure.

Recently, a coding agent was asked to fix a one-character typo in a README file (true story, bear with me). It listed the repository’s open issues, created a branch, committed the change, and opened a pull request, consuming more than 21,000 input tokens along the way. One keystroke of value, a small novel’s worth of compute.

Stories like this are no longer entertaining; in 2026 they are line items on an ever expanding bill. The economics of a standard LLM deployment and the economics of an agentic deployment share almost nothing beyond the vendor invoice, and most companies discover the difference after the architecture decision has already been made.

One Request Is Never One Call

Gartner’s March 2026 analysis puts agentic workloads at 5 to 30 times more tokens per task than a standard chatbot, and typical production agents land between ten and twenty model calls for a single user request.

The arithmetic worsens with ambition. RAG pipelines ship large context windows with every query. Always-on monitoring agents scan logs, inboxes, and market data around the clock, consuming compute whether or not a human is watching. These background workloads barely existed in enterprise budgets two years ago. Today they represent a growing share of inference spend that most finance teams never approved, because nobody itemized it.

At 10,000 users, a single agentic feature can run between $150,000 and $750,000 per month. The pilot that looked viable at fifty users was measuring a different system. The code did not change, and neither did the model. Volume changed, and volume turned out to be the entire story.

There is even a rough way in which this hits. Analyses of failed agent deployments place the cost cliff between 500 and 5,000 users, the range where cloud API pricing stops making sense and teams face a forced migration to self-hosted GPUs they never planned for. One documented startup watched its unit economics invert between 700 and 1,000 concurrent users and killed the product.

The system worked technically. It failed as a business, and the failure was baked in at the whiteboard stage.

Read more

Engineers working with AI tools now spend more hours reviewing generated code than writing new code. Digital Applied’s Q1 2026 survey of 2,847 developers recorded the crossover, with review overtaking writing as the largest AI-assisted time sink after writing held a four-hour lead as recently as 2024.

Over the same period engineering managers have moved in the opposite direction. They are now more technically hands-on than they have been in a decade as Agentic AI lowers the barrier to direct code contribution.

Both trends meet in the middle of the org chart. The division of responsibilities between engineer and EM was doing structural work that few leaders ever named, and coding agents are dissolving it with nothing arriving to replace it.

The Job Descriptions Stopped Matching the Work

Look at how a senior engineer on an AI-heavy team actually spends a Tuesday. She kicks off two agent runs before standup, reviews a stack of generated pull requests mid-morning, fixes a prompt configuration that started producing flaky tests, and switches contexts across three tools before lunch.

The Pragmatic Engineer’s 2026 survey of over 900 engineers and engineering leaders captured exactly this. Engineers orchestrate more and context-switch more often, managers can be more hands-on, and the survey’s authors flagged the conclusion themselves. The engineer and manager roles are becoming similar.

Managers are converging from the other side. An EM can now ship a fix or prototype a feature between one-on-ones, and plenty of them do. The technical distance that used to accumulate after two years in management has stopped accumulating.

The Old Division Was the Long Pole

The classic split assigned implementation quality to engineers and gave managers allocation, priorities, and people. AI broke this in both directions.

When agents produce the majority of a feature’s code, the person directing them is making allocation decisions. Which tasks go to the machine, which stay human, how much scrutiny each output deserves. That used to be manager territory. And when a manager merges their own AI-assisted fix, they have re-entered the codebase their role was designed to stay out of.

What Role Confusion Costs

One client I worked with last year had three teams improving the same AI workflow at the same time.

Engineering upgraded the model to reduce latency. The AI team refined prompts and retrieval settings to improve answer quality. Operations updated the business rules the agent was expected to follow.

Each team shipped good changes. Each team achieved its own goals. A month later, overall accuracy had dropped.

No single change caused the problem. It was the interaction between all three. Everyone was optimizing their part of the system, but no one owned the system itself.

We uncovered it during a retrospective and established a single owner for end-to-end evaluation, along with shared metrics across the teams.

That problem happened to be visible. Most aren’t.

A Tilburg University study of Copilot adoption in open-source projects found core developers reviewing 6.5% more code while their original output dropped 19%. Stack Overflow’s 2025 survey found 45% of developers citing time-consuming debugging of AI-generated code as a top frustration. And in a March 2026 SmartBear survey of 273 software leaders, 70% said application quality had already degraded as AI accelerated development.

Those numbers tell a consistent story. Code production is accelerating faster than organizational ownership.

When an engineering manager merges agent-generated code and a production incident surfaces two weeks later, who owns the postmortem? The engineer who approved the pull request? The team that tuned the prompts? The platform team that selected the model? The product manager who defined the workflow?

Teams without a clear answer pay for AI twice. Once for the tokens, and again for the coordination overhead of figuring out whose responsibility the output became.

A Redesign That Fits on One Page

The fix requires less machinery than most reorgs. In my work with engineering teams adopting agentic workflows, four decisions cover most of the confusion.

Name one accountable reviewer per code surface. Agent-generated pull requests get a single human owner, assigned by code area and written into the CODEOWNERS.md file. That owner can be an engineer or an EM. What matters is that exactly one name appears, so accountability for quality survives the increase in volume.

Give manager code contribution explicit rules. If an EM ships code, it goes through the same review path as everyone else’s, and its scope stays bounded. Prototypes, internal tooling, and spikes work well. Critical-path features do not, because a manager who owns production code has become an engineer with a reporting-line problem.

Put orchestration in the engineer job description. Hours spent directing agents, writing evals, and maintaining prompt configurations should count as engineering work in performance reviews. If promotion criteria still reward hand-written lines, engineers will optimize for the old job while the actual work goes unmeasured.

Rebuild the EM role around what AI left behind. Stakeholder negotiation, cross-team decisions, career development, and the judgment calls agents consistently fumble. Those responsibilities gained value as everything around them got automated, and a manager whose calendar reflects that is doing the redesigned job instead of competing with their own engineers for the review queue.

Then revisit the whole arrangement quarterly. The tools are changing fast enough that a role definition written in January describes a different workflow by June.

An org chart is a claim about how work gets done. Each quarter the chart goes unedited while the work underneath changes, the claim gets a little less true. The teams outperforming with AI-assisted delivery share one habit that costs nothing to copy; they wrote down what changed. Engineers who manage agents, managers who touch code, and one name on every review.

Role convergence turns out to be a design problem, and design problems reward the leader willing to name them.

For about fifteen years, the force that decided where enterprise value collected had a name. Dave McCrory called it data gravity in 2010, and the idea aged well.

Applications drift toward data because moving data is slow, costly, and risky. Whoever controlled the data layer controlled the decisions that stacked on top of it, from analytics to applications to budgets.

That logic still holds, but what reaches for your data has changed. The dashboards and pipelines that used to sit beside the warehouse are giving way to agents, and an agent does not stay next to the data. It runs on some platform, reasons over whatever it can reach, and moves results between systems on its own.

The platform you pick to run your agents is taking over the position the data layer used to hold. It is becoming the thing that owns the relationship with your data.

That is a bigger decision than it looks, and most teams are making it without noticing.

Once those walls go up, the position is expensive to win back, which is what makes this a decade-long bet and not a procurement round.

Agent gravity is the newer force

Tomasz Tunguz called this shift “agent gravity” in a recent essay. The argument runs parallel to the old one. Agents demand enormous compute, that compute is a large and growing business, and the platforms hosting agent workloads will fight to keep them. The more agents and data flowing through a platform, the heavier its pull.

Agents are turning into the main surface through which people and systems touch enterprise data.

An employee asks an agent instead of opening a dashboard. A customer interacts with an agent instead of a form. Other automated systems call an agent instead of hitting a database directly.

Once that becomes the default path, the platform running the agent sits closer to the value than the platform storing the data. Proximity to the work now accumulates more leverage than custody of the bytes.

Why running the agents is where the moat forms

Running an agent is expensive, and that expense is the point. Inference at scale, orchestration, memory, tool calls, retries, and the guardrails that stop an agent from doing something costly all burn compute, and compute is the business these platforms are in.

Tunguz has written separately about the harness, the orchestration and control layer that turns a raw model into something an enterprise can trust. That harness is where the hard engineering lives now. Whoever owns it owns the relationship with everything the agent reads, writes, and moves.

This is why the platform decision outlasts the model decision. Models will keep leapfrogging each other on every leaderboard.

The harness around them, the place your agents are configured, governed, and run, is sticky in a way individual models never were. I have seen an arrangement like this start as one convenient integration and end, two years later, with the bulk of a company’s analytical work running somewhere nobody picked on purpose.

The doors are already closing

Incumbents understand the dynamic, and they are not waiting for you to notice it. In April, Microsoft removed the compatibility mode that let Power BI query Databricks metric views through the standard connector, which broke the reports that relied on it (the release notes state it without ceremony).

At Build 2026, Microsoft positioned Fabric as the data platform for its Copilot and agent ecosystem, wired Fabric IQ into Microsoft 365 Copilot, and shipped Agent Skills that let agents build models and reports directly on governed Fabric data.

The behavior repeats across the field. Snowflake pushes Cortex, Google leans on BigQuery, and every one of them wants your agents reasoning over data inside its own walls.

The friction a vendor removes inside its own stack becomes friction everywhere else. That asymmetry is the gravity well, and it is built on purpose.

The question worth asking

This reframes what an evaluation should measure. Benchmark scores age in weeks, and a model that tops a chart today will sit mid-table by the next release. Tuning a ten-year decision around this quarter’s numbers misreads the timeline.

The operators I work with tend to ask a sharper question once they see the mechanics.

Which layer of the stack will own the relationship with our data over the next five to ten years? A company that stores its customer data in one system and runs its agents through another has already answered that question, whether it meant to or not. It handed the relationship to whoever controls the agent runtime, and it did so without holding a meeting about it.

Three checks separate a deliberate choice from an accidental one. First, can your agents read and write across platforms, or does every convenient path keep everything inside one vendor? Second, when an agent copies or moves data, who holds the audit trail and the off switch? Third, if you had to move your agent workloads to a different platform in three years, what would break, and what would it cost?

When the honest answer to the third question is that nobody has ever priced it, the platform has already priced it for you.

Make the bet on purpose

None of this argues for paralysis. Single-vendor stacks are convenient, and convenience earns its keep when a team is small and shipping fast. The narrower point is the one worth holding onto. The choice of where your agents run is compounding into control over your data, and that control is hard to win back once a vendor has built the gravity well around it.

Pick with open eyes, and price the exit before you need it. A platform decision you file under tactical has a habit of turning into the most strategic call you made all decade.

There is a version of the enterprise AI story told in board meetings, and a version told in weekly standups, and the uncomfortable truth is that both are accurate.

In the board meeting, the chart goes up and to the right. Adoption is up, usage is up, time-to-first-output is down. Agents are running, employees are experimenting, and the company appears to have crossed from AI aspiration into Agentic AI execution. Nobody is lying. The chart is real.

In the standup, sales says the AI-generated account briefs are useful, after someone verifies the facts. Support says the agent drafts good responses, except the policy-sensitive ones, which is to say the ones that matter. Engineering says coding agents accelerate scaffolding, and that senior engineers just lost most of a sprint untangling an AI-generated migration that passed review and failed in staging. Operations says the workflow agent handles the happy path, and that when it doesn’t, someone spends two days reconstructing what the agent actually did, which systems it touched, what data it relied on, why it made the call it made, because nothing was built to replay it.

The temptation is to decide one group is wrong: the executives are high on their own supply, or the operators are foot-dragging. Neither. They are looking at different layers of the same system. Executives see the application layer. Operators live in the integration layer. And only one of those layers makes it onto the dashboard.

That is the abstraction error at the center of most enterprise AI programs, and it is worth being precise about, because the companies that fix it first are going to be very hard to catch.

Production Is Not Absorption

Dashboards measure what is easy to instrument: users, prompts, drafts, summaries, agent runs completed, time saved to first output. None of these numbers is fake. All of them measure the same thing, AI production, and production was never in doubt.

Producing more output faster is the entire point of the technology. Celebrating it is like celebrating that the printing press produces pages.

Read more

A supervised AI tool hands you a draft and waits. You read it, you edit it, you decide whether it ships. But an agent does not wait. It reads a support ticket, queries a database, updates a CRM record, sends a few emails, and schedules a follow-up, finishing most of that before anyone looks at the outcome.

The model underneath can be identical. The oversight problem is a different beast entirely.

Most governance frameworks running in production were written for the first kind of system. They assume a person checks each output before it carries consequences, so the controls cluster around the moment of approval. That design holds up well when AI generates something and stops. It comes apart the moment an agent chains actions together across systems and accounts, where each step sets up the next and no one is standing at the gate.

The data shows how wide this gap has grown. In McKinsey’s 2026 AI Trust Maturity Survey, only about 30 percent of organizations reached maturity level three or higher in strategy, governance, and agentic AI controls, even as deployment footprints kept expanding. Technical capability is racing ahead. The oversight structures meant to keep it accountable are lagging, and the distance keeps widening.

Sequences change what oversight has to catch

The reason supervised guardrails fall short with agents comes down to how the two systems fail. A supervised tool fails at a single point. It produces a bad draft, a person catches it, and the cost stops there.

An agent fails along a path. It misreads one input, acts on that reading, and every action after it inherits the error. By the time anyone notices, the agent has touched five systems and the original mistake is buried three steps back.

This is why security and risk concerns now sit at the top of the list of barriers to scaling agentic AI, cited by close to two-thirds of respondents in the same survey. The worry has shifted from capability to control. Teams want to know what happens when an agent does something it was never explicitly told to do, and whether anyone can reconstruct the chain of events well enough to undo it.

A governance framework built for agents has to account for the sequence rather than the endpoint. That means defining the boundaries of what an agent may touch, building checkpoints into the path instead of bolting them onto the final output, and deciding in advance what happens when an agent operates outside its intended scope.

Make every agent decision traceable

When an agent acts across systems, the most valuable thing you can have afterward is a record of why it did what it did, which input triggered which action, and which decision produced which outcome. Without that trail, an incident becomes a forensic exercise with no evidence, and the team is left guessing at a system that already moved on.

McKinsey’s survey found that the rate of AI incidents has held steady at roughly 8 percent, yet confidence in how organizations respond to them has dropped. Close to 60 percent of respondents who experienced an incident rated their organization’s response as no better than satisfactory. Incident frequency has stayed flat. The ability to trace, explain, and contain those incidents has fallen behind the complexity of the systems creating them.

Traceability is an engineering problem before it becomes a compliance one. It means logging the agent’s reasoning and actions in a form a human can reconstruct, designing systems so a single decision can be traced back to its trigger, and building the audit trail into the architecture instead of adding it after something goes wrong.

Agents that cannot explain themselves are agents you cannot govern.

Governance belongs in engineering before it reaches compliance

A lot of organizations are waiting for regulation to tell them what good looks like. That instinct is understandable, and it is also fragile. The EU AI Act’s high-risk obligations for stand-alone systems were originally set to apply in August 2026, and in May 2026 EU lawmakers reached a political agreement to push most of them to December 2027.

Transparency rules still land in August 2026, but the headline deadline that many teams were planning around moved by more than a year.

This is the core problem with running agentic oversight off a regulatory calendar. The calendar reflects political negotiation, and it tells you nothing about how your specific agents fail, what they can reach, or how you would catch them when they drift. Those are engineering questions, and they get answered well only by people who understand the architecture.

What to ask next

Agentic governance comes down to a few honest questions:

• What can this agent reach?

• What does it do when it gets something wrong?

• How can you trace any outcome back to the decision that caused it?

A team that can answer these has already built the things a large enterprise customer or regulator asks for: an agent with bounded access, a defined response when it gets something wrong, and an audit trail someone can actually follow.

Now think back to the agent I described at the beginning. It read the ticket, queried the database, updated the record, and sent the emails before anyone looked at the outcome. In this scenario, oversight waits until the end of that chain.

Apply these questions to understand how the workflow could look different. Then the control sits inside the system instead of at the final output, put there by the people who built it before the agent ever runs.

Your agents are already acting across live systems, and the only governance that protects you is the kind you build into how they work.

Fourteen AI initiatives on a single roadmap, governed by one steering committee, measured against one set of success criteria. Half are automating existing workflows to protect margins. The other half are building capabilities the company has never offered before. Meanwhile, the budget, risk framework, and quarterly check-in schedule remain stagnant.

This is what most enterprise AI portfolios look like right now. And it explains why so many of them feel stuck.

The two halves of that portfolio are fundamentally different games. One is about protecting what already works. The other is about building what comes next. Each requires different ownership, different timelines, different metrics, and different tolerance for ambiguity. Running them as a single strategy is like training for a marathon and a sprint on the same schedule. The structure guarantees that one of them suffers.

What Most Organizations Miss

McKinsey’s Global Tech Agenda 2026 found that the CIOs delivering measurable value have made a specific shift. They’ve moved technology from a cost center to what McKinsey calls a “value creator,” embedding AI and data directly into operating models.

But the research surfaced a clear divide between organizations that are simply modernizing their technology estate and those that are rewiring for competitive advantage.

That divide maps to a pattern I keep running into with enterprise leaders. The companies actually moving forward are playing two distinct games at once:

1.With defense, they’re using AI and Agents to protect the core business. Automating manual workflows, tightening operational efficiency, reducing cost structures that have been bloated for years.

2. On offense, they’re building new capabilities. New products, new revenue streams, new ways of reaching customers that weren’t possible eighteen months ago.

Most organizations don’t have a mental model for this split. They’re either in pure cost-cutting mode or chasing growth, and the AI and Agentic AI strategy simply reflects whichever game the board happens to be pressuring this quarter.

What Defense Actually Looks Like

Defensive AI and Agent targets processes you understand well, with outcomes you can measure in months and risk profiles you can model. Automated claims processing. Intelligent document extraction. Predictive maintenance on equipment that’s already generating revenue.

The success criteria are clear. Faster cycle times, lower error rates, reduced headcount for routine tasks, better margins on existing lines of business. The value case is arithmetic, and the ROI conversation is relatively straightforward.

What Offense Actually Looks Like

Offensive AI builds capabilities that didn’t exist before. You’re not optimizing a known process. You’re testing whether a new process should exist at all.

These projects look like using AI to enter adjacent markets with personalized products, or building recommendation engines that fundamentally change how customers discover what you sell, or creating internal decision-support tools that give your operators information advantages competitors don’t have.

The success criteria are murkier. You’re measuring learning velocity, market signal, and option value. The ROI conversation is harder, and the organizational patience required is significantly higher.

When Efficiency Eats Innovation

When companies run offense and defense under the same governance structure, the defensive projects almost always win the resource fight.

Defense gets measured on efficiency, cost reduction, and operational reliability. The governance is tighter and accountability sits with operational leaders who own the processes being improved.

Offense gets measured on learning rate, market validation, and strategic optionality. The governance is much lighter, and the timelines are longer.

Overall, defensive projects are easier to justify, easier to measure, and easier to get approved. So offensive projects get deprioritized because they can’t compete on the same ROI framework.

The result is a portfolio that looks busy, but only plays one game. The company gets more efficient at what it already does while falling behind on what it could become. The board sees cost savings and assumes the AI and Agent strategy is working, but nobody’s building anything that changes the company’s competitive position.

The Diagnostic

If you’re running AI and Agent initiatives right now, here’s a quick test. Look at your active portfolio and sort every project into one of two columns. Column one: protecting existing revenue and margin. Column two: building something you’ve never had before.

If you can’t sort them cleanly, your strategy is probably conflated.

The companies losing ground on AI and Agents aren’t necessarily the ones spending too little. They’re the ones who never made the split visible, never assigned ownership to each side, and ended up with a portfolio that defaults to whichever pressure is loudest.

Making the split explicit is the first step toward making it work.

McKinsey’s research found that more than 80% organizations are not yet seeing a tangible impact on enterprise-level EBIT from AI and Agents. This suggests that while adoption is broadening, most companies are still struggling to turn AI and Agents into scaled financial results.

But there is an important piece of the story that is missing. A separate analysis of 140 enterprise AI implementations found that 77% of failures were organizational in nature, with technical issues like model performance, data quality, and integration complexity accounting for less than a quarter.

Your org chart is the first system AI has to survive before it reaches a single customer or workflow, and these five structural mistakes consistently prevent it from getting there.

1. Your Chief AI Officer Reports Nowhere Near the P&L

The 2026 AI & Data Leadership Executive Benchmark Survey found that 38.5% of companies have now appointed a Chief AI Officer or equivalent, but there’s almost no consensus on where that role sits. Reporting lines are split across technology, business, and transformation leadership, with no dominant model emerging and no clear pattern connecting any one reporting structure to better outcomes.

That fragmentation carries real downstream consequences. When AI leadership reports into a CTO or CIO function, the role tends to optimize for infrastructure and tooling decisions rather than business impact. When it reports into a transformation office, it gravitates toward strategy decks and governance frameworks that rarely survive contact with operational reality.

Neither path connects AI or Agents directly to revenue, margin, or operational throughput, which means the person nominally responsible for AI results often has no line of sight into the metrics that define them.

2. Your AI or Agent Team Lives in IT Instead of in the Business

When AI or Agent capability gets housed inside the IT department, it inherits IT’s entire operating model, meaning projects get scoped through a service request lens, prioritization follows the IT backlog, and success gets measured in uptime and deployment velocity rather than business outcomes.

This is a fundamental structural mismatch. AI is a business capability that requires technical infrastructure, and the distinction matters because AI initiatives that start with a business problem and work backward toward the right technical approach tend to survive past the pilot stage, while initiatives that start with a model and go looking for a use case tend to stall indefinitely.

Organizations running AI teams embedded within business units, or at minimum co-located with business leadership, consistently outperform centralized IT-led models on both adoption and value delivery.

3. Your Steering Committee Owns Accountability for Nothing

AI steering committees are one of the most popular governance structures in enterprise AI programs, and they’re also one of the least effective.

The typical setup includes senior representatives from multiple functions who meet monthly to review progress, offer guidance, and align priorities, but in practice, these committees almost always devolve into a venue for status updates where no actual decisions get made.

The root issue is accountability without power. Steering committees rarely control budget allocation, staffing decisions, or deployment timelines, which means they can recommend changes but have no mechanism to compel them. When an AI initiative hits an organizational obstacle (and every one does), the committee discusses it, documents it, and then waits for someone else to resolve it, creating a governance layer that absorbs time without reducing friction.

Research on AI governance maturity from McKinsey’s 2026 AI Trust Maturity Survey reinforces how widespread this gap is, with only about 30% of organizations reaching a maturity level of three or higher in governance, even as their technical and data capabilities continue to advance. The organizational decision-making apparatus simply hasn’t kept pace with the technology it’s supposed to govern.

4. You Built AI Skills in One Team and Called It Done

Concentrating AI talent in a single team feels efficient at first, but the problems with this approach emerge at scale. When every AI initiative has to flow through the same team, that team becomes a bottleneck.

This pattern appears so frequently in enterprise organizations that it has earned a name in organizational design circles. It’s called the Center of Excellence trap.

The CoE starts as a strategic asset and gradually evolves into a capacity constraint that chokes the very pipeline it was built to open. A CIO article from late 2025 described the resulting dynamic well, noting that business units inevitably branch off on their own when the central AI team can’t keep pace, creating fragmented and ungoverned efforts scattered across the company with no shared standards or oversight.

The more sustainable model is capability distribution. Instead of hoarding AI expertise in one group, the investment goes into building baseline AI literacy and applied skills across functions. This allows the central team to shift from doing the work to enabling others to do it by providing tooling, standards, training, and quality guardrails while the business units own execution and outcomes.

5. Your Center of Excellence Has No Authority to Make Anything Stick

This is the inverse of mistake four. Some organizations do build a Center of Excellence with a genuine mandate to drive AI adoption across the enterprise, staffing it well, giving it a clear charter, and expecting it to set standards for how AI gets developed, deployed, and monitored. Then they forget to give it any enforcement power.

What follows is predictable. The CoE publishes best practices that business units ignore, develops governance frameworks that project teams route around, and recommends tooling standards that departments override. Without budget influence, or the organizational standing to block non-compliant deployments, the CoE becomes an advisory function that advises no one in particular and enforces nothing at all.

This is a design failure at the leadership level. A CoE with clear standards but no enforcement mechanism creates the illusion of governance while fragmented, uncoordinated AI adoption continues underneath it.

The Real Infrastructure Problem

These five mistakes share a common thread. They all treat AI as something that can be added to an existing organizational structure without redesigning how decisions get made, who owns outcomes, and where authority actually lives.

AI underperformance in most organizations traces back to an org chart that was built for a different kind of work and never updated to reflect how AI-driven operations actually need to function.

The companies capturing real returns in 2026 are the ones willing to redesign reporting lines, redistribute decision rights, and place AI leadership where it can actually influence how the business operates on a daily basis.

If you’re reviewing your AI strategy this quarter, start with the org chart. The structure you’re running determines the ceiling of what AI can deliver, and right now, most ceilings are set lower than anyone realizes.

AI and Agentic AI costs have a way of looking reasonable right up until they aren’t.

The early pilots run on contained use cases with limited traffic, so the numbers stay small and nobody questions the architecture behind them. Then the product scales. Teams start layering inference calls into features that weren’t in the original cost model, and the spend starts compounding in places nobody is watching.

By the time finance flags the invoice, the architecture driving those costs is already embedded in production and expensive to change. A Gartner survey found that more than 90% of CIOs say managing cost limits their ability to extract value from AI at scale.

The problem is rarely any single API call. It’s the accumulation of decisions that were never designed to hold up under real production volume. These four levers address that directly. Each one targets a different layer of the cost structure, and together they give you a system that stays predictable as usage grows.

1. Right-Size Model Selection to Task Complexity

The fastest way to cut AI costs without changing outcomes is to stop sending every request to your most capable model. Most production AI workloads follow a clear pattern where a small percentage of requests require deep reasoning while the majority involve extraction, classification, or short-form responses that a lighter model handles just as well.

A model routing layer evaluates each incoming request and directs it to the appropriate model based on complexity, confidence thresholds, or task type. Simple queries go to smaller, faster, cheaper models. Only the requests that genuinely need frontier-class reasoning get routed to the expensive option.

Read more

We’ve all read about the AI rollouts that go awry. Tools get purchased, training gets scheduled, an adoption campaign goes out, but within two months the usage curve flattens because nobody in the organization can answer a simple question:

What specific problem are we solving, and how will we know we solved it?

I’ve spent years leading teams from both an engineering and product management perspective, so I’ve seen from the trenches why this obvious question can get skipped. The urgency to “adopt AI” pushes companies straight into tool selection and training programs while the harder, slower work of defining which problems are actually worth solving never happens.

The Missing Discipline

A recent Harvard Business Review study by Amanda Pratt and Melissa Valentine examined AI adoption at a major tech company and surfaced a finding that should reframe how every operator thinks about this problem.

It was no surprise to me that the area most correlated with successful, sustained AI adoption turned out to be product management, not prompt engineering or technical fluency. The disciplines that mattered most were defining which problems are worth solving, designing structured experiments, and integrating solutions into the way work already happens.

These findings line up with what I’ve observed across dozens of AI and Agentic AI engagements. The companies where AI actually takes root are the ones that approach adoption with product discipline, starting with a specific workflow, identifying a measurable friction point, building a small test, and evaluating results before scaling anything.

Two Companies, Two Approaches

Consider the difference between two real patterns I see repeatedly in enterprise AI and Agentic AI work.

1) Company A purchases an AI platform, negotiates an enterprise license, builds a prompt library, and launches a change management campaign complete with lunch-and-learns, weekly tip emails, and a login dashboard to track “adoption.” After three months, a handful of power users have integrated the tool into their workflows, and everyone else has moved on.

2) Company B takes a different path. Before selecting any tool, they run a structured problem-definition process across three business units. Each unit identifies its highest-friction workflow, documents the current state in detail, and defines what a measurable improvement would look like. Only then does the team evaluate which AI capabilities (if any) could address those specific problems. They run 30-day pilots with clear success criteria, and when two of the three pilots produce measurable gains, those two scale while the third gets killed early, saving months of wasted effort.

One of those pilots, for example, targeted a procurement approval workflow that averaged nine days from request to sign-off. The team mapped every handoff, identified two steps where AI-assisted document review could eliminate manual bottlenecks, and set a target of reducing cycle time to under four days. After the pilot, cycle time dropped to three and a half days. That result gave leadership concrete evidence to fund a broader rollout in procurement, and the specificity of the success made it easy to communicate across the organization.

Company B spent less money, took slightly longer to get started, and ended up with AI embedded in actual workflows producing actual results. Company A spent more, moved faster, and ended up with an expensive tool that sits mostly unused.

Why Problem-Definition Keeps Getting Skipped

The rise of AI has put immense pressure on companies to try to move fast. But the problem-definition process feels time consuming and slow. On the other hand, buying a tool and launching a training program feels like jumping quickly into action.

There’s also a structural gap. Most organizations assign AI adoption to IT or to a newly created “AI team” that reports to the CTO. Those teams are good at evaluating technology. They’re less practiced at the product management work of scoping problems, defining success metrics, and designing experiments within business workflows they don’t own. The people closest to the workflows (operations leads, department managers, senior ICs) rarely get pulled into the problem-definition phase because the initiative is framed as a technology project, not a workflow improvement project.

Velocity without direction is just expensive motion. The organizations I work with that have the strongest AI adoption results are the ones that invested the first four to six weeks in problem definition and a Data Story / IP Moat audit before evaluating a single vendor. That initial patience created clarity that made everything downstream faster, from tool selection to pilot design to scaling decisions.

The Diagnostic Question

If you want to know whether your AI or Agentic AI adoption effort has legs, ask one question across every team that’s supposed to be using AI. Can they answer, specifically, what problem they’re solving and how they’ll know if they’ve solved it?

If the answer is vague (”We’re using AI to be more efficient”) or circular (”We’re adopting AI because we need to adopt AI”), the rollout is already in trouble. Clear problem statements are the leading indicator of whether AI adoption will stick or stall.

The companies that bring product management discipline to AI adoption, with defined problems, scoped experiments, and honest evaluation, end up with AI embedded in their actual operations. Everyone else ends up with a line item on the budget and a login dashboard nobody checks.

You would never bring a new hire onto your team without performance feedback, escalation paths, or a way to know when they’re struggling. Yet that’s exactly how most organizations deploy AI agents. MIT Sloan and BCG’s 2025 research found that 76% of executives now describe agents as coworkers rather than tools, but almost none of them are managing agents that way. They ship the agent and move on.

Deciding to call your agents “coworkers” is easy. Setting up the feedback loops, escalation paths, and failure signals that actually make one is where teams stall. It’s almost entirely an infrastructure problem, and these are the six pieces most teams skip.

1. Evaluation Frameworks

A working agent and a reliable agent are two different things. Evaluation frameworks give you the ability to measure the difference before your users discover it for you. This means building structured test suites that run against your agent’s outputs on a regular cadence, scoring for accuracy, relevance, and task completion across a range of realistic scenarios.

Good evaluation suites include both deterministic checks (did the agent call the right tool with the right parameters?) and judgment-based scoring (was the response actually useful to the person asking?).

The key is that evaluation has to be continuous, running in CI/CD pipelines and against live traffic, because agent behavior shifts as underlying models update and data distributions change. LLMs, the technology that undergirds agents, are at their core probabilistic in nature, which means there is an often opaque statistical distribution that can shift over time, which affects performance and accuracy.

Anthropic’s engineering team has written publicly about maintaining evaluation suites as living artifacts, with dedicated teams owning the infrastructure while domain experts contribute tasks and run the tests themselves.

2. Fallback and Escalation Logic

Every agent will encounter situations it cannot handle. The question is whether you’ve decided in advance what happens next, or whether the agent improvises.

Fallback logic defines the boundaries. When confidence drops below a threshold, when a tool call returns unexpected data, when the task exceeds the agent’s defined scope, the system needs a predetermined path. That path might route to a simpler deterministic process, a different model, or a human operator. Escalation logic layers on top of that by adding severity awareness.

Without explicit escalation tiers, every failure gets the same treatment, which means either everything gets flagged (and humans stop paying attention) or nothing does (and real problems slip through). The organizations successfully scaling agents build these paths before deployment, treating them as load-bearing architecture.

3. Monitoring for Drift

AI agents degrade quietly. Model updates, shifts in input data, changes to upstream APIs, seasonal variation in user behavior. Any of these can erode agent performance without triggering a single error.

Drift monitoring tracks the gap between how your agent performed when you validated it and how it performs now. This includes statistical monitoring of output distributions, latency tracking across individual tool calls, and automated quality scoring against baseline benchmarks. In practice, effective drift detection requires capturing baseline metrics during your evaluation phase and then running the same scoring pipeline against production traffic on an ongoing basis. When scores diverge from your baseline by more than an acceptable margin, you have a concrete signal to investigate rather than a vague feeling that things seem off.

4. Human-in-the-Loop Checkpoints

Full autonomy sounds efficient until you realize what it costs when the agent is wrong. Human-in-the-loop checkpoints create structured moments where a person reviews, approves, or redirects agent output before it reaches the end user or triggers a downstream action.

The design challenge is placement. Too many checkpoints and you’ve built an expensive autocomplete system. Too few and you’ve handed off accountability to a system that can’t actually hold it. The right approach maps checkpoints to consequence.

Low-risk, reversible actions can run autonomously. High-stakes decisions, anything involving money, legal exposure, or customer-facing commitments, need a human gate. As agents take on more complex workflows, these checkpoints also become your training data pipeline. Every human correction is a signal about where the agent needs improvement, but only if you’re logging it (which brings us to the next point).

5. Logging for Auditability

When an agent makes a decision, you need to be able to reconstruct exactly how it got there. Full execution logging captures the chain of reasoning, tool invocations, retrieved context, intermediate outputs, and final actions across every run.

This serves three purposes simultaneously:

First, debugging. When something goes wrong, you need the trace, not a guess.

Second, compliance. Regulated industries require demonstrable decision trails, and even unregulated ones are moving in that direction.

Third, improvement. Logged executions become the dataset you use to identify failure patterns, tune prompts, and build better evaluation suites.

The tooling for this has matured significantly. OpenTelemetry-based tracing, structured span capture, and production replay capabilities now exist across multiple frameworks. The infrastructure cost is low relative to the cost of operating an agent you cannot inspect.

6. A Defined Handoff Protocol

Agents rarely operate in isolation. They pass work to other agents, to human operators, to downstream systems, and occasionally back to the user. Every one of those transitions is a potential failure point.

A handoff protocol specifies what information transfers with the task, what context the receiving party needs, what constitutes a successful handoff versus a dropped one, and who owns the outcome after the transition.

This gets more complex in multi-agent systems where one agent’s output becomes another agent’s input. If the first agent summarizes a customer issue and strips out a critical detail before passing it along, the second agent makes a decision on incomplete information. Neither agent has failed individually, but the system has failed completely.

Without this kind of structural clarity, you get the agent equivalent of a game of telephone. Context gets lost between steps, responsibilities blur, and when something fails mid-workflow, nobody can pinpoint where.

The Management Layer You Can’t Skip

These six elements share a common thread. They’re all infrastructure that exists to manage the agent after it’s built.

The agent itself, the model, the prompts, the tool integrations, that’s maybe 40% of what a production deployment actually requires.

The other 60% is the system that keeps the agent honest, visible, and recoverable when things go sideways.

Organizations that treat agent deployment as a build-and-ship exercise will spend the next six months doing manual cleanup on failures they could have prevented. The ones that invest in this management layer first will find that their agents get better over time instead of quietly getting worse.

The technology is mature enough. The question is whether your operational infrastructure is ready to match it.

Serving a 7B parameter model costs roughly $0.0004 per 1,000 tokens. A frontier model like GPT-5 charges up to $0.09 for the same volume. That’s a 200x spread on per-token cost, and at production scale, it compounds into the kind of line item that makes CFOs start asking uncomfortable questions.

Yet most enterprise AI strategies still start in the same place. Frontier model API, default configuration, build everything on top.

I’ve heard the same reasoning for this decision countless times. The plan is to start here, and optimize later. But “optimize later” rarely happens. The API dependency becomes load-bearing, and switching costs quickly accumulate. More often than not, teams discover much too late that 70-80% of their inference calls are handling structured, repeatable tasks that never needed frontier-class reasoning in the first place. Meanwhile, a fine-tuned small model handles all of it at a fraction of the cost, often with better accuracy on the specific domain, and without the vendor dependency.

The question worth asking before you architect anything isn’t “which model is most powerful.” It’s whether the task even requires that power.

The Compounding Cost Problem

The per-token price gap between frontier and small models tells only part of the story. The real damage happens at volume.

Gartner’s analysis found that agentic AI workflows consume 5 to 30 times more tokens per task than standard chatbot interactions. When your agents are running thousands of structured, repeatable tasks per day, each one burning frontier-priced tokens, monthly inference bills can scale from manageable to alarming before anyone notices. A system handling 50,000 daily agent tasks on frontier APIs accumulates costs that a finance team will eventually flag, and “but the model is really smart” isn’t a satisfying answer when 80% of those tasks are pattern execution.

Read more

Cursor crossed $2 billion in annualized revenue in early 2026. The team that built it? Roughly 300 people. Gamma, the AI presentation platform, hit $100 million ARR with about 50 employees and has been profitable for over two years. Midjourney generates hundreds of millions in annual revenue with a team you could fit in a mid-sized conference room. Lovable reached $100M ARR in eight months with 45 people.

Meanwhile, the median private SaaS company generates about $130,000 per employee. Five years ago, $100K was considered a reasonable benchmark. At scale, the best traditional SaaS companies were proud to reach $300K.

The gap between these numbers tells you something specific about how these companies are built. All four companies I mentioned initially have something in common beyond the headcount math.

From the first hire, they were built around AI as a core operator, with every workflow, every role, and every system designed on that assumption. The label for this is AI-native.

And for founders and executives running $5–30M ARR companies right now, the gap between AI-native operations and everyone else is a competitive timeline that is already shrinking.

What “AI-Native” Actually Means at the Operational Level

The phrase gets thrown around loosely, so let me be specific. An AI-native company designs its workflows from scratch around what AI can do. Every process, every role, every system assumes AI as a core participant from day one.

This is fundamentally different from what most companies do, which is take their existing workflows and add AI tools to them. The distinction matters because the architecture of your operations determines the ceiling of your efficiency.

Consider how a traditional SaaS company handles content. A marketing team writes briefs. Writers produce drafts. Editors review. Designers format. A project manager coordinates the whole thing. Five or six people touch every piece of content before it ships.

An AI-native company designs that workflow differently from the start. AI generates first drafts from structured inputs. A single editor shapes the output. Distribution happens programmatically. The entire pipeline might involve one or two people instead of six, and the throughput is three to five times higher.

Multiply that across customer support, engineering, sales enablement, onboarding, and internal operations. The compounding effect explains how Cursor runs at $6 million per employee while companies with similar revenue require ten times the headcount.

Why Retrofitting Existing Operations Fails

The instinct most established companies have is to layer AI tools onto what already exists. Buy a few licenses, integrate a copilot, maybe automate some ticket routing. This feels productive. It rarely moves the needle in a meaningful way.

The problem is structural. Your existing workflows were designed around human throughput. Your org chart reflects that design. Your hiring plans, your meeting cadences, your approval chains, your reporting structures all assume that humans do the work and other humans coordinate that work.

Bolting AI onto this foundation creates an awkward hybrid. AI generates a draft, but then it still goes through the same five-person review chain that existed before. AI triages support tickets, but the staffing model hasn’t changed to reflect the reduced load. The tool saves twenty minutes per task, but the organizational overhead around that task stays identica

The Realistic Options for Established Companies

If you’re running a $5–30M ARR company, you probably aren’t going to tear everything down and rebuild from scratch. That’s fine. But pretending the efficiency gap will close on its own is a mistake with a deadline.

Here’s what actually works for companies that aren’t starting from zero.

Start with one workflow, redesigned from zero. Pick your highest-volume, most repeatable process and redesign it from scratch with AI as the primary operator. Don’t optimize the existing process. Design the new one as if the old one didn’t exist. Customer onboarding, content production, and first-line support are common starting points because they’re high-volume and have clear inputs and outputs. The goal is to prove to your own organization what redesigned throughput looks like before you try to scale the approach.

Hire for the new architecture. The next time you open a role, ask whether the function that role serves could be restructured around AI instead. This doesn’t mean replacing people. It means designing the role so one person with AI leverage can do what previously required three. The companies generating $2M+ per employee didn’t get there by giving existing employees AI tools. They built teams where every person operates as a force multiplier.

Measure the right ratio. Track revenue per employee quarterly. If you’re below $150K and growing, you’re adding headcount faster than you’re adding efficiency. That was fine in 2020. Today, it means you’re falling behind the curve that AI-native competitors are setting. For context, top-quartile SaaS companies now generate $350K-$700K per employee, and the AI-native outliers are running at five to ten times that range.

Accept that partial adoption produces partial results. A company that redesigns 30% of its operations around AI-native principles will capture meaningful efficiency gains. A company that gives everyone a ChatGPT license and calls it transformation will not. Architectural commitment drives the outcome here. Tool selection alone never has.

Sequence your investment around leverage. Most companies adopt AI where it’s easiest to implement. The better approach is to start where the ratio of human labor to repeatable output is highest. That’s usually operations and fulfillment, where the actual throughput gains live.

The Clock Is Running

The revenue-per-employee gap between AI-native companies and everyone else keeps widening. Gartner projects a wave of companies generating $2M+ per employee by 2030, and the leaders are already well past that mark.

For operators and founders at the $1–5M stage, this isn’t a future problem. Your next funding round, your next hire, your next operational decision is happening in a market where competitors might need one-fifth the headcount to deliver the same output.

The companies that approach this as an architectural challenge will adapt. The ones running a tool-buying exercise will learn the hard way that efficiency at this scale comes from how you build, from how you design the work itself.

Gartner projects that over 40% of agentic AI initiatives will be abandoned by 2027. Reading that, a reasonable person might conclude that there is an inherent issue with the technology.

However, I know from my own experience building agents that when done correctly, they deliver.

The failure pattern we keep hearing about has nothing to do with model quality or infrastructure maturity. It’s that organizations have no single agreed-upon definition for their own data.

Different departments define the same terms differently, and agents consume whatever definition they hit first at 10x the speed any human team would. Humans reconciled those gaps in quarterly meetings and footnotes. Agents just produce confident, expensive wrong answers.

The real fix requires a role that most companies haven’t named yet.

When “Revenue” Means Different Things

Humans have always tolerated semantic drift inside organizations. If marketing and finance calculate revenue differently, they reconcile the gap in quarterly meetings or bury it in footnotes. The cost of ambiguity stayed low because humans processed data slowly enough to catch the mismatches.

AI agents don’t reconcile by themselves. They ingest whatever schema they can access, apply whatever definition they encounter first, and produce output that sounds authoritative regardless of whether the underlying logic holds.

The confidence of the output actually makes the problem worse, because stakeholders trust polished summaries more than they trust raw numbers.

Read more

A year ago, agents could reliably handle about an hour of autonomous work. Tasks like summarizing a document or running a data pull. Useful, but contained. You could bolt those tasks onto existing workflows without changing anything structural.

That window is closing fast.

METR, the AI evaluation research organization, published findings last year that reframed how I think about planning horizons:

• The length of tasks that frontier AI agents can complete with 50% reliability has been doubling approximately every seven months.

• In the 2024-2025 period, the pace accelerated to roughly every four months.

• Agents that managed one-hour workflows in early 2025 will be handling full eight-hour workstreams by late 2026.

An eight-hour workstream is a fundamentally different unit of work than a one-hour task. And most companies have no operating model for that.

The Staffing Problem Nobody’s Solving Yet

When an agent handles a one-hour task, it fits neatly inside your existing org chart. But when an agent handles an eight-hour workflow, you’ve crossed into project-level work.

This raises questions your org chart wasn’t designed to answer. Who scopes the work? Who reviews quality at intermediate checkpoints, not just at the end? If the agent makes a judgment call four hours in that sends the remaining four hours in the wrong direction, whose problem is that?

Most executives are still thinking about AI as a task-level tool, something that makes individual contributors faster. The planning shift required here goes deeper. If an agent can own a full workday of output, you’re making staffing decisions, not automation decisions. And staffing decisions cascade. They affect headcount planning, team composition, project timelines, and how you think about accountability for deliverables.

Consider a concrete example. A three-person analytics team currently handles weekly reporting, ad hoc data pulls, and quarterly business reviews. At the one-hour level, agents might handle the data pulls. The team stays intact, just faster. At the eight-hour level, an agent can own the entire weekly reporting cycle, from data extraction through visualization to narrative summary. Now you’re looking at a different team shape entirely. Maybe two analysts and one workflow architect who designs and monitors the agent pipelines. Same output, different organizational logic.

Tomasz Tunguz has been writing about this transition from the venture side. He’s running 31 agent tasks a day through his own workflows and watching software engineers manage 15 parallel AI workstreams through GitHub. The throughput numbers are real. But throughput without organizational redesign just creates a different kind of mess.

What Breaks When You Map Agent Capabilities Onto Human Structures

Here’s where most companies get stuck. They take their existing team structure, identify tasks within that structure, and hand those tasks to agents. That works fine at the one-hour level. At the eight-hour level, you start hitting structural mismatches.

Human team structures assume certain things. People accumulate context over days and weeks. They build judgment through repeated exposure to similar decisions. They escalate ambiguity upward. But agents don’t operate on any of those assumptions. They start fresh each time (unless you architect context persistence). And they’ll confidently proceed through a six-hour workflow on a flawed assumption made in hour one.

That’s a critical insight for anyone planning around agent-length workflows. The longer the workflow, the more you need architectural guardrails, not because the agent is incompetent, but because compounding errors over eight hours of unsupervised work can waste the entire output.

Designing Work Around Agent-Length Workflows

So what actually changes in practice? Three things.

First, decomposition becomes an engineering discipline. When you’re handing off an eight-hour workstream, the quality of your work breakdown determines the quality of the output. Vague briefs that a senior employee could interpret and correct on the fly become expensive failures when an agent executes them literally for a full workday. The skill shifts from “manage the person doing the work” to “architect the specification precisely enough that autonomous execution succeeds.”

Second, review cadence matters more than review depth. A single end-of-day review of eight hours of agent work is a recipe for rework. The Deloitte research on agentic AI adoption found that organizations succeeding with agent workflows redesigned their review processes around intermediate checkpoints, not final deliverable review. The parallel in software engineering is obvious. You don’t wait for the entire codebase to be written before doing a code review. You review at the pull request level. Agent workflows need the same kind of incremental quality gates.

Third, accountability has to be redesigned, not just reassigned. When a human employee produces bad work, the feedback loop is straightforward. When an agent produces bad work after eight hours, the accountability question splits in several directions. Was the specification wrong? Was the workflow architecture missing a checkpoint? Did the person who scoped the work understand what the agent could and couldn’t handle? These are systems questions, not performance questions. And they require a different management muscle than most organizations have built.

The Planning Horizon Question

Companies that wait until agents can reliably own full workdays before restructuring will be rebuilding their operating models under time pressure. Companies that start now, rethinking work decomposition, review cadences, and accountability frameworks, will have the organizational muscle in place when the capability arrives.

The point here goes beyond headcount replacement. The unit of work you’re managing is about to change scale. A hiring plan built around task-level automation looks very different from one built around project-level agent staffing. The team structure that works when agents handle one-hour tasks won’t hold when they handle eight.

The businesses that get this right won’t be the ones with the best AI models. They’ll be the ones that redesigned their operations to match what agents can actually own.

…

Nick Talwar is a CTO, ex-Microsoft, and a hands-on AI engineer who supports executives in navigating AI adoption. He shares insights on AI-first strategies to drive bottom-line impact.

→ Follow him on LinkedIn to catch his latest thoughts.

→ Subscribe to his free Substack for in-depth articles delivered straight to your inbox.

→ Watch the live session to see how leaders in highly regulated industries leverage AI to cut manual work and drive ROI.

Every workflow has invisible seams, steps that only function because a human with ten years of context fills the gaps.

Most companies don’t notice these gaps because the process works well enough and the entire human’s job is not documented, step-by-step (an unreasonable expectation, of course). What usually happens in these cases is people route around the broken handoff, apply judgment where the documentation runs out, and quietly absorb complexity that was never formally accounted for.

Oftentimes, humans supporting and filling gaps is great when humans run the workflow. But as the use of AI agents begins to rise, things start to change and each one of these gaps become places where the agent fails and never picks up.

Drop an agent into a workflow built on informal human compensation, and the agent will execute the process exactly as written. Which means the real question is whether the workflow itself was ever designed to run without a human quietly holding it together.

For most companies, the answer is no. And that means the work needs to start with workflow redesign.

Why Pilots Succeed and Scaling Breaks

Pilots work because a small team compensates for every gap the agent can’t handle. Scale via Agents and technology removes that team. What’s left is a workflow designed for humans, now being executed by software with zero tolerance for ambiguity.

Agents don’t adapt to broken handoffs. They don’t infer ownership when it’s unclear. All they do is follow the process as defined.

If the process is being held together by informal knowledge and human workarounds, the agent will expose every seam.

About 90% of the function-specific AI use cases that hold real transformative potential are still stuck in pilot, according to McKinsey. The problem is process and workflows, not technology. High-performing AI companies are roughly three times more likely to redesign workflows from scratch rather than layer agents onto what already exists. The redesign is where the real value lives.

What Workflow Redesign Actually Looks Like

Redesigning a workflow for agents means answering four questions at every stage of the process.

Question 1: Which steps can an agent fully own?

These are tasks with clear inputs, defined outputs, and minimal need for contextual judgment. Data extraction. Standardized formatting. Pulling records from structured sources. If the step can be described as a contract (this input produces this output, within these constraints), an agent can own it.

Question 2: Which steps require a human decision point?

Anywhere the process involves evaluating trade-offs, exercising risk tolerance, or making a call that depends on relationships or institutional context. These steps don’t disappear when agents arrive. They become more visible, because the agent will stop and wait rather than guess.

Question 3: Where does the agent hand back?

The handoff points matter more than most teams realize. A poorly defined handoff creates the same ambiguity problem that broke the original workflow. Every transition between agent and human needs an explicit output contract. The agent delivers a specific artifact, in a specific format, with a clear expectation for what the human does next. Vague handoffs like “the agent prepares a draft for review” just move the ambiguity to a different part of the chain.

Question 4: What does the output contract look like at each stage?

This is where most redesigns fail quietly. Teams define what the agent does but skip defining what “done” looks like at each step. Without an output contract, downstream steps inherit uncertainty, and the compounding effect makes the whole workflow fragile.

This Is an Org Design Decision

Most conversations about AI agents stay in the technology lane. Which model, which framework, which vendor.

But deploying an agent into an existing workflow is an organizational design decision. You’re changing who does what, where decisions get made, and what information flows where. That makes it a structural change to how your operation runs, and it deserves the same rigor you’d apply to any reorg.

Skipping the redesign means the agent will faithfully execute a process that was already broken. It will do it faster, at scale, and with none of the informal corrections that made it barely work before. Every workaround your team normalized over the years becomes a failure point. Every undocumented decision becomes a gap in the chain.

The companies pulling real value from agents share one thing in common. They were willing to look at a workflow that “works fine” and admit it only works because humans have been compensating for design flaws the org stopped noticing years ago

The issue was architectural: they’d given the LLM complete autonomy over execution decisions, and LLMs simply aren’t built to provide deterministic control at that level.

Gartner predicts that over 40% of Agentic AI projects will fail to reach production by 2027. The difference between systems that scale reliably and those that collapse under real-world conditions comes down to whether you separate reasoning from execution.

Where Failures Actually Originate

The latest LLMs demonstrate remarkable reasoning capabilities. They can break down complex tasks, weigh tradeoffs, and generate sophisticated action plans. The problem emerges when organizations confuse reasoning capability with execution reliability.

LLMs are probabilistic pattern matchers trained on text. These characteristics propagate to Agentic AI systems built on top of LLMs. They excel at understanding context and generating plausible responses. But they struggle with deterministic execution, maintaining consistent behavior across edge cases, and guaranteeing the same output given similar inputs. Even when they appear to be well understood during pre-production testing and simulation.

Zenity Labs found that classifiers fail when inputs take unexpected paths through activation space. The classifier works perfectly on inputs it recognizes, but novel paths (even semantically similar ones) can produce completely different classifications. The same dynamic applies to Agentic AI: systems trained and tested on known scenarios encounter unfamiliar patterns in production, and their responses become unpredictable.

When you let the LLM make execution decisions directly, you’re betting that production will only present scenarios the model has learned to handle reliably. That bet fails more often than teams expect.

Full Autonomy Creates Unpredictability

In production environments, Agents don’t receive clean, well-formatted inputs. They encounter ambiguity, partial information, conflicting signals, and edge cases that fall outside training distributions.

Consider an Agent tasked with processing refund requests. In testing, requests follow predictable patterns. In production, you get:

• Requests that qualify for refunds but use non-standard phrasing

• Borderline cases where policy interpretation matters

• Situations requiring escalation that don’t match trained escalation triggers

• Inputs that combine multiple issues in ways the model hasn’t seen

When the Agent has full autonomy, it must decide in real-time which action to take. Small variations in input phrasing can trigger entirely different action sequences. Run the same ambiguous request twice, and you might get different outcomes. This happens not because the model is malfunctioning, but because probabilistic systems don’t guarantee determinism.

This behavior compounds across interactions. An Agent processing hundreds or thousands of decisions daily will inevitably encounter scenarios that push it outside reliable operating ranges. Without external controls, there’s no mechanism to catch these situations before they produce incorrect actions.

The Control Layer Solution

The Control Layer architectural fix separates what LLMs do well (reasoning) from what they do poorly (deterministic execution).

In this model:

1. The Agent analyzes the situation and proposes an action

2. A control layer validates whether that action is permitted

3. Only validated actions execute

The control layer uses rule-based logic that encodes business constraints, compliance requirements, and operational boundaries. When the Agent proposes an action, the control layer checks:

• Does this action fall within permitted operations?

• Do the action parameters meet safety constraints?

• Are required conditions satisfied?

• Does the user context allow this operation?

If validation passes, the action executes. If not, the Agent receives feedback and can propose an alternative. Taking time to address these questions as a team, distill it into requirements, and then work with engineering to distill them into a Control Layer architecture is a core mitigation strategy for these business risks.

This architecture maintains the Agent’s flexibility while ensuring predictable boundaries. The Agent can still reason about complex scenarios and adapt to novel situations. The control layer ensures that adaptation happens within defined limits.

The Right Level of Control

Building systems that consistently do the right things matters more than maximizing autonomy.

Control layers define boundaries that let Agents operate confidently within them. Inside those boundaries, Agents can be remarkably flexible, adapting to novel scenarios and learning from outcomes. The boundaries simply ensure that adaptation doesn’t violate business requirements or create unpredictable behavior. It also gives you a backstop to monitor and close feedback loops, slowly improving the system over time so less escalations occur.

Organizations that skip this step typically discover the need for controls after production failures. By then, retrofitting governance becomes significantly harder than building it from the start (akin to putting a genie back in a bottle).

The systems that succeed in production share a common architecture: they separate reasoning from execution, maintain clear decision boundaries, and enforce validation before actions reach production systems. That architectural choice (more than model selection, training approach, or testing strategy) determines whether Agentic AI delivers predictable value or unpredictable failures.

• They hallucinate

• They don’t understand your specific business context without extensive prompt engineering

• Once your data enters external systems, monitoring who accesses it becomes extremely difficult

A hybrid AI model helps to combat these issues. Instead of retrofitting security onto external systems, you build with two distinct layers from the start. You run a proprietary core trained on your fragmented internal data. You use generalized LLMs as utilities for non-sensitive tasks.

Different problems require different tools, and your most valuable data deserves more than API-level protection.

How a Hybrid Model Works

A hybrid setup operates with two distinct layers, each designed for different types of work.

The Core Proprietary Model handles everything that requires institutional knowledge or contains sensitive information. This layer gets trained or fine-tuned specifically on your internal data. The fragmented information sitting across databases, documentation systems, and tribal knowledge that actually runs your business. You deploy it privately (air-gapped, on-premises, or in tightly controlled infrastructure). You own it, govern it, version it.

The Generalized LLM Layer functions as a utility, similar to electricity or cloud compute. Use it for broad reasoning tasks, general drafting, summarization, anything that doesn’t touch sensitive context.

Regulated customer data, competitive intelligence, and process IP stay in the proprietary core. General business tasks that could happen anywhere go to the utility layer.

Why This Works

It Eliminates Prompt Engineering Overhead

When your core model already understands domain-specific terminology, business rules, and institutional patterns, the prompt complexity drops. You stop spending cycles explaining your context in every interaction.

In my work with companies moving domain-specific work to fine-tuned internal models, I’ve seen prompt engineering overhead drop by 50-60%. The model knows product SKUs, understands compliance requirements, recognizes org structure. Questions that would require three paragraphs of context setup with ChatGPT work with a single sentence.

It Turns Fragmented Data Into an Asset

Fine-tuning a model on this distributed knowledge creates something actually useful. A unified intelligence layer that has ingested and made sense of information across silos. The model becomes a practical interface to knowledge that was previously locked away.

It Preserves Privacy Without Killing Usability

The user experience can look nearly identical to ChatGPT. What changes is what sits behind that interface.

The sensitive operations happen in infrastructure you control:

• Customer PII never touches OpenAI’s servers

• Competitive analysis stays internal

• Compliance teams can audit exactly what data moves where

Once data enters a big tech system, monitoring who accesses it becomes extremely difficult. Current privacy regulations create genuine liability when you can’t track data lineage.

It Reduces Black-Box Provider Risk

The hybrid model limits exposure by keeping your most sensitive information completely separate from external systems. You’re not trusting a third party to respect your anonymization or to maintain proper access controls. The data simply never leaves your environment.

When you own the core, you control the governance model, the retention policies, the access logs. When you rent utilities, you’re only exposing information you’d be comfortable seeing anywhere.

When to Own, When to Rent

The decision framework comes down to three questions.

Does this task require institutional knowledge? If the answer depends on understanding your specific processes, products, or customer context, it belongs in the proprietary core. If any competent professional could handle it with general knowledge, it can run through the utility layer.

What’s the sensitivity level? Regulated data, competitive intelligence, unreleased product details all stay internal. General business writing, research summaries, basic analysis can use external LLMs.

What’s the cost of being wrong? If a hallucination or data leak creates regulatory exposure, reputational damage, or competitive harm, you need the control that comes with ownership. If mistakes are cheap to catch and fix, utility models work fine.

Most enterprises find that 20-30% of their AI workload truly requires the proprietary core. The rest can run on general utilities, where you benefit from continuous model improvements without the maintenance burden.

Building for the Long Term

The hybrid approach requires upfront investment. You need to train or fine-tune models, set up private deployment infrastructure, and establish data pipelines. But the payoff is control over your most sensitive operations and ownership of the intelligence you develop.

The risks of sending enterprise data through external systems are very real: data leakage, compliance violations, and loss of competitive intelligence are real outcomes that enterprises can’t afford. The hybrid model eliminates these exposures by keeping sensitive work on infrastructure you control.

Once you’re operational, your most frequent queries run at marginal cost. Every interaction with your proprietary model generates data you can use to improve it. The intelligence stays with you.