Ramp published a report this week showing that there is comparatively limited usage of Fable, while Opus’ usage has jumped very quickly. Simultaneously, with the flood of open-source model releases, we’re hearing more and more about teams interested in domain-specific post-training.

That reminded of us this post from last year: The direction an increasingly intelligent and abstracted model goes in might be different from the direction that an application needs models to go in. How you handle that divergence is critical question for application builders, and the argument is increasingly leaning in favor of open-source intelligence.

GPT-5 is the first major model release from OpenAI that we didn’t immediately jump to put in production at RunLLM. To be clear, we tried, but the models that were released simply didn’t make sense for us to prioritize amongst the many other things that we could spend our time on, and the models that we would have normally prioritized didn’t show meaningful improvements on our benchmarks.

Reflecting on why, we have a few key takeaways about the evolution of LLM providers over the last couple years and where the API-first products are headed. Our headline takeaway is that there are early signs of a divergence between pure LLM and inference and smarter LLM-based “products.” As application builders, we have a distinct need for control over what the underlying models are doing, so LLM inference is more appealing to us than a more tightly coupled product. GPT-5 is the first major model release that feels like a product rather than a pure LLM, which is part of the reason why we feel GPT-5 doesn’t quite meet our needs.

• Consumer apps vs. developer APIs. We — like many others — complained about OpenAI’s product UX in the days of the model picker with 7+ models. Giving a user that many options when they wanted to run a simple query didn’t make much sense. The new UX is dramatically better, even if it had a rocky rollout (and even if GPT-5 has some room for improvement compared to previous iterations like o3). Developer APIs on the other hand are different: Application builders are going to make thoughtful, informed, often empirical choices about what options to pick, so having a variety of options is good. The options let an application builder trade off cost, latency, and quality depending on the complexity of each task they’re doing. That said, the previous model line ups were more intuitive — thinking about the reasoning effort in gpt-5 and how that compares to a smaller model size’s output is not immediately obvious to us, and reasoning models in general seem to be higher variance (more on that below). The migration guide on the OpenAI docs helps, but we didn’t find the analogs to be drop-in replacement in our experiments. As you can tell, there’s no simple answer here, but there’s definitely room for improvement.

• Reasoning applications vs. agentic applications. We noticed a wave of GPT-5 based product announcements after the launch, though fewer than before. What sticks out to us is that there are applications which benefit from throwing more compute in a single LLM call for a particular task — examples like the ones Box CEO Aaron Levie posted around document processing fall into this bucket. For these use cases, switching to a more powerful reasoning model makes perfect sense. However, for agentic applications like RunLLM, improving quality often comes from composing multiple LLM calls rather than having one big call to a powerful model. In fact, we’ve found that reasoning models — dating back to o1 — tend to increase variance in our performance in a way that our customers (and therefore our team) find unnerving. As a result, we’ve yet to put a reasoning model in our production inference pipeline; instead, we primarily rely on non-reasoning LLMs that are scoped to very narrow tasks and composed into more powerful pipelines. Some of this might be fixed with better prompt tuning, but the variance in latency and output has led us to be more conservative than before with these models.

• Open-weight vs. closed-weight. With all of the quality open-weight models being released over the last 6 months, it feels increasingly viable to construct high-quality applications that rely on open models. The main motivation is a sense that there isn’t any confusing logic hiding under the hood. Again, this wasn’t a concern 1-2 years ago, but as the model providers try to build more verticalized model products, it feels like a concern that might begin to emerge over the next few model releases. This will especially feel true if OpenAI moves to deprecate the GPT-4.1 family of models, which we still rely on heavily and would be quite sad to lose. Being forced to switch to a reasoning model for a simple task like filtering out irrelevant questions is not our first choice. The non-reasoning version of GPT-5 is too one-note to replace everything we do, especially since the smaller model is not available via API. For the reasons described above, we find that reasoning models aren’t (yet) the best fit for us. In that world, relying on open-source models from unbiased infrastructure feels like a safer bet — or at least moving to lean more heavily on model providers like Google, that seem to be closer to exposing “just the model.” We’re not necessarily confident that this will be an issue for us, but we’re cognizant that this is a risk area for our product today. If Google and Anthropic feel the need to follow suit with OpenAI’s latest releases, there will be a strong motivation to switch to more decentralized inference services that are transparent with model behavior rather than more abstracted services.

• Model inference vs. model products. Our bigger picture observation on this front is that pure inference seems to be diverging from a model as a “product.” In 2023 and 2024, an open-source inference product provided a similar output to a vertically integrated model provider API — just with different weights. It’s starting to feel like that is changing, with the model providers trying to own more of the stack and include more functionality in the system itself. Reasoning and tool use are the two most obvious examples of this. Both are great from a consumer perspective, but they don’t fit our needs as application builders. That is going to affect the direction of the development of future models. If the major model providers are in an arms race to develop the best model for consumer applications, that might start to diverge from what a model that’s useful as a composable building block in applications would look like. We don’t know enough about the development roadmaps at the frontier labs to say for certain, but we’ve certainly heard engineers from these teams say things like “a good enough model will solve enterprise problems” (which we strongly disagree with). Products becoming increasingly vertically integrated and opinionated would undoubtedly push us towards open models.

As with most things in AI over the last few years, we’re speedrunning the hype cycle of past platform shifts. What’s unique about this situation is that OpenAI has managed to build both a consumer and enterprise business over just a few years, and unlike Google or Amazon as they built out their cloud businesses, both sides of OpenAI’s business rely on the same foundation.

OpenAI might very well see its consumer product as the end-all-be-all that will replace other applications. We disagree, and so does likely everyone building AI applications. The question is whether OpenAI enables its enterprise customers to continue to rely on its API products or pushes them away with too much abstraction.

With the news of Airtable’s acquisition this past week, analyses of the previous generation of SaaS companies have started flying around the internet. Our opinions haven’t changed dramatically, so we thought we’d bring back this post from 6 months ago.

Stay tuned — we’ll be ramping back to regular content in the next couple weeks!

Projections about the demise of the SaaS industry have reached a fever pitch over the last few weeks. The general belief seems to be increasingly that there’s no moat in software anymore. Consequently, we have seen the stock prices of massive, entrenched incumbents take a significant beating.

The bear case for software-as-a-service goes something like this: As coding agents improve, the build-versus-buy calculus shifts permanently. An engineer at any company can now pick up a coding agent, build a prototype that meets the company’s specific needs, iterate on feedback, and deploy a bespoke tool that provides more value than a generic SaaS platform at a fraction of the cost.

The source of this concern is our collective recent experience with the improving quality of coding agents. We have all had Claude Code or Cursor scaffold an entire application from scratch for a few dollars in tokens. When you see something genuinely useful built in minutes, it is easy to assume the multi-billion-dollar incumbents are doomed. And if an individual can build a prototype quickly, surely a startup can build a strong offering with a few months and a few engineers?

As with most hype cycles, however, the truth lands in the middle. While there are massive opportunities for disruption, the SaaS business model is not going to evaporate overnight. In our view, the distinction comes down to a quick survival checklist. If you meet one of these criteria, you are likely safe. If you miss on all of them, you should be worried:

1. Are you a system of record?

2. Do you do more than help humans automate a single workflow?

3. Are you mission-critical?

The Physics of Data Gravity

Data moats have long been the holy grail of enterprise software. Consumer products like Google or Instagram win on user behavior data, and enterprise titans like Snowflake or Datadog are powerful because they are systems of record. These companies do not just power workflows; they house years of historical data that companies have imported and structured.

Despite all the massive technological shifts in the last few years, the physics of data have not changed. Moving massive amounts of data is expensive, risky, and slow. This means that, as has been the case for fifteen years, the major cloud providers will continue to make their margins on networking and egress. The cost of moving data in and out of a third-party service — or your own cloud — remains incredibly high.

Beyond the operational cost, there is the issue of operational stickiness. If a product is collecting data for a core operational purpose, it becomes a load-bearing wall in the company’s architecture. You do not pay Datadog or Snowflake eight figures a year because storing logs is a nice-to-have feature. You pay them because when a mission-critical issue happens at 3:00 AM, you need a proven way to investigate, pinpoint, and fix the issue.

But realistically, your business will continue to exist if Datadog goes down for a little while. Things get much more difficult when we’re talking about truly mission-critical software.

The Mission-Critical Tautology

Even when companies don’t have the most interesting data, they’re likely safe if they are critical to the operations of their customers – the kinds of products that your business literally wouldn’t exist without. The obvious examples are Workday and Salesforce.

There is a certain recursive logic here: These companies are safe because they are big, and they are big because they are safe. No matter how shiny a rapidly scaffolded payroll system looks, a VP of HR is not going to risk payroll failing on a Friday morning. A CRO does not care how many new automations a startup offers if it means moving away from a Salesforce instance they have spent a decade customizing to their exact sales and revops motions.

When enterprises make these decisions, they are not just buying software – they are also offloading risk. A tech-forward company like Google or Meta certainly has the technical talent to build an internal payroll system. The question is not whether they can build it, but whether they want to own the risk of running it. Once they do the calculus, dedicating hundreds of engineers to a non-core area of expertise rarely makes sense.

By contrast, non-mission-critical software is in the danger zone. Analytics is the prime example. Traditionally, writing code for dashboards and plumbing database queries was prohibitively expensive to democratize.

Just last week at RunLLM, we built an internal dashboarding system following a conversation about product metrics. We used Python and Plotly to connect to internal systems and pull customer data via our CRM’s API. If this dashboard goes down for a day, it is annoying, but it does not halt our operations. More importantly, because the cost of building it with a coding agent was so low, and the upside of allowing our VP of Sales to modify it himself was so high, the option to buy a traditional BI tool never even entered the conversation.

The Workflow Moat Narrows

Pure workflow software is where we’re currently the most bearish. By workflow software, we mean anything that’s focused on connecting the dots between existing tools that either have data gravity or mission-criticality. The poster child on X for this is PagerDuty. At its core, PagerDuty processes data stored in another product’s telemetry store, determines if a threshold was met, and alerts an on-call engineer. This is connecting the dots between Datadog and Slack. While PagerDuty does much more in practice, that primary workflow is what most customers buy.

These are exactly the kinds of integration code workflows being replaced by agents today. The threat here is not just that agents will automate incident management or sales outbound; it is that new players or internal teams can build superior user experiences that blend human and agent capabilities from the ground up. The most interesting part of this comes from the fact that the solutions can (and should) very easily be customized to match each team’s workflow. These agent-first systems will be dramatically more valuable because they save human hours, easily displacing legacy systems that are human-first and currently scrambling to bolt on AI features as an afterthought.

The Ops Gap

The undiscussed concern underlying all build it with Claude Code software is operations. The reality of software operations and reliability is unfortunately still incredibly complex and oftentimes dwarfs the time taken to build something useful. The gap between building and operating is actually growing because as coding becomes a commodity, the relative cost of infrastructure and security grows.

The internal dashboard we built recently is a perfect example of this. It took about an hour to iterate on the core dashboards themselves with Claude Code. It then took about four hours to deploy it in Google Cloud so that it was properly authenticated behind our company’s SSO, set up to auto-update with new commits, and connected with the right credentials.

This gap will likely persist. The simple reason why is that Python looks the same no matter where you work, but infrastructure is dramatically different at every company. There are not yet great ways to deploy cloud software without getting into the weeds of Kubernetes clusters, networking permissions, and security + compliance. None of those things matter when you are prototyping locally, but they are the only things that matter when you are operating production software at scale.

Wrapping Up

The rumors of the demise of modern software are greatly exaggerated, but the nature of what makes software valuable is quickly changing. For the last decade, SaaS companies could survive by being a slightly better UI for a human workflow. That’s no longer defensible.

The defensibility remains exactly where it has always been: at the intersection of data and risk. If you own the system of record, the physics of networking and the high cost of data egress will protect you. If you own a mission-critical operational process, the corporate aversion to risk will protect you.

The real danger is for the middle layer of the stack — the products that have built businesses around the friction of human work. As the cost of generating code and automating workflows drops toward zero, the value of that software must move elsewhere.

This does not mean incumbents are invincible. It just means that the disruption will not come from a simple internal tool or a weekend project built with a coding agent. To win in this new environment, startups must figure out how to best manage the data, reduce the operational risk, and close the gap between a prototype and production-grade reliability.

As concerns about coding agent overuse, budget overruns, and the ultimate impact on productivity mount, we’ve found ourselves wondering about the economics of coding agents. Coding agents can generate code faster than engineers working alone, but faster production is not the same as better economics. The fact that you can generate more code doesn’t mean that you should. To be clear, this isn’t a question of writing code by hand – all code will be generated by agents – but what tasks you pick, what models you use, and how you manage your agents.

To illustrate the point, let’s take a somewhat obvious example. A group of four can drive from SF to LA in 6 hours and pay, perhaps $100 total for gasoline. That same group of people can fly from SF to LA in 1 hour and pay $200 each for airfare. Both are valid choices and depend on the urgency of the trip, what they’ll do once they arrive, and their price sensitivity.

Engineering leaders are grappling with the fact that the economics of producing software, managing engineering time, and ultimately building products have changed dramatically. If you get your parameters wrong, you’ll be paying a lot of money to build the wrong thing very fast.

The economics of software production

If we rewind to the halcyon days of 2022, the way most engineering teams budgeted was by having a set headcount and allocating those cycles to whatever the highest priority projects were. Assuming the team delivered as expected, you could roughly say that you were spending $N a quarter to produce the software you shipped that quarter.

Today, you can spend something like $2-3N to create the same software in, perhaps, a week. While it’s hard to quantify, this is definitely a more efficient process – a cost increase of 2-3x and time improvement of about 10x is some pretty easy math for any project that you can look at.

For a real world example, consider the recent port of Bun from Zig to Rust. The blog post detailing the post says that the port took about 11 days and cost $165k at API pricing for work that would have taken a “small team of engineers a full year.”

This introduces a totally foreign dimension to software budgeting. To be clear, this is a huge net improvement. A small team, say engineers, working for a year will easily cost you $600k. The fact that you can do this task for ~25% the cost doesn’t mean that you should do it. If the task was done with the same number of tokens on Sonnet but required 2x as much human input, that would cost ~$50k of tokens combined with an extra 10 days of salary. The reality probably isn’t that simple, but it’s illustrative of the new dimension of cost in producing software.

Until recently, we haven’t really had to ask this question about software production, but coding agents have completely changed the game. Engineers and engineering teams now have to grapple with questions about which model they should use, what level of speed and accuracy they need, traded off against a willingness to use a cheaper model that requires more guidance.

Ultimately, the economics of software production are going to increasingly have to be tied to the value of what’s being produced, not just the time it takes to produce it. Which brings us to the humans using the models…

The economics of engineering time

Measuring engineering productivity has historically been extremely hard. As an industry, we long ago ruled out silly metrics like lines of code, commits made, or tasks closed. In their place, we’ve mostly just agreed that the good engineers are the ones that everyone else on the team agrees are the good ones. DORA metrics can point at the function of an engineering team but don’t tell us much about individual engineering performance.

Recently, engineering leaders have started measuring engineers on token usage. This is unequivocally a bad idea, but it highlighted a key concern around how you think about whether engineers are using their time well and delivering on the right things. As is now becoming common wisdom, what you build matters dramatically more in a world where you could theoretically build anything.

In practice, the 2x2 between good vs. bad productivity and low vs. high token use has 4 valid quadrants. In other words, token use does not correlate positively or negatively with engineering productivity. Just because an engineer burns through a lot of tokens, that doesn’t tell you very much about what’s getting done – they might be mismanaging context windows, regularly patching issues created by poor prompting, or undoing and redoing a lot of work because of poor prompting. Just recently, an engineer on our team accidentally burned through ~$300 tokens on Claude very quickly by mistakenly pinning Fable. That engineer is generally very good but obviously didn’t suddenly get 10x better than everyone on that day. Simultaneously, we have plenty of examples of engineers who are very judicious about token use because they plan carefully, give precise instructions, and invoke the agent when they’re confident in which direction they want to head.

There’s no easy answer here. Measuring engineering productivity has genuinely gotten dramatically harder in the past 6 months because we’ve introduced a whole new dimension that simply didn’t exist before. You shouldn’t crown an engineer because they burned through a bunch of tokens, and you shouldn’t fire them either.

Engineering decisions & product decisions

The technology is new, and these aren’t easy decisions to make. In the stylized example above, we suggested that doing the same task with Sonnet instead of Fable would have been possible with 2x as much human input – that might be true, or Sonnet might have very quickly gone off the rails and generated totally useless results. Perhaps $165k + 10 days of time is the cheapest possible implementation with the current state of the art. It’s impossible to know otherwise until you actually try. Unfortunately, trying is expensive.

While token costs dropped rapidly in 2023-24, the rate of change has decreased over the last year – and the rate of token consumption has skyrocketed. Managing this tradeoff has gotten harder as we’re now all balancing model choice with effort level and task scope. Given that it’s new, every engineer is going to find this challenging, but this will be a skill that engineering teams develop over time with practice.

In the meantime, you need to keep a close eye on your token use.

Theoretically, we should have a stellar AI agent for every problem in our lives by now. The talent is there, the capital is certainly there, and the models are increasingly capable. And yet, the results are lopsided. Why is it that we have agents that can prospect for sales leads and answer support tickets accurately, but we don’t seem to be able to consistently generate high quality slides?

The simplest explanation might be complexity. Easier problems (e.g., answer a support question) naturally get solved first, and more open-ended problems like slide generation require more effort. That doesn’t quite hold up: Coding is obviously not a simple application area, and yet coding agents are some of the best that we have today – in fact, they are improving faster than any other single agent use case.

How did this happen? Ease of adoption enabled data collection at scale that in turn helped coding agents improve rapidly.Every developer could switch to Cursor in 5 minutes without any approval. That created a data flywheel (more on this below) that allowed the Cursor team to build a better application experience over time – to the point where our whole team now swears by Cursor’s Composer model for code generation.

The combination of technical complexity and adoption difficulty creates an interesting 2x2:

You might be tempted to think that being in one of the “easy to adopt” quadrants is the holy grail – after all, who doesn’t want more data to build better models? That is certainly a valid way to build a business, but the trap is that easy to adopt also means easy to displace. Hard to adopt products have their own data moat: Once you’re embedded in an enterprise, you learn about how that company works in a way that makes your product incredibly hard to replace.

Whichever quadrant you fall into, data is your only moat.

Easy to adopt, easy to solve

Easy to adopt and easy to solve is the most obvious quadrant to work in. It didn’t take an incredible amount of foresight to see back in 2023 that consumer search on Google would be replaced by custom answers to every question that a user has – whether it was finding a nice fact or providing healthcare advice. This has been the bread-and-butter use case for the foundation model providers and plenty of new entrants (e.g., Perplexity, You.com) flocked to these use cases as well.

The “easy to solve, easy to adopt” quadrant is a value trap. If the barrier to entry is low for you, it’s non-existent for frontier labs (or more likely, they’ve already built it). Given that these are the “obvious” use cases, they’re the ones for which the existing chat applications will see the highest volume of usage. That means that – whatever the use case is – OpenAI, Google, and Anthropic are gathering millions of data points to improve their models in these areas. Last week’s release of ChatGPT Health feels like an obvious step in this direction. Beyond data access, the model providers can also subsidize costs and leverage their massive user bases to learn any new application area quite quickly. In short, you very likely will get crushed by the model providers.

An interesting side note is that loyalty is quite low in this quadrant – we all use multiple chat agents depending on the use case, and unlike with the web search market, everyone seems to be on relatively equal footing. If dominant brand leaders do emerge, we’d place our bets on the model providers.

Easy to adopt, hard to solve

Why did coding – ostensibly one of the hardest problems to solve! – see such rapid progress? Most importantly, it is because adoption was easy – you could see a ton of value by pasting a snippet of code into ChatGPT back in 2023, and Cursor quickly made that much easier even though quality was limited early on. Since every engineer typically has the freedom to choose their own IDE, switching from IntellIJ or VSCode to Cursor wasn’t a crazy lift. Once it was in place, it also had a very fast feedback loop – a software engineer might generate code with Cursor tens or hundreds of times a day. That created a data flywheel: Every accepted or rejected suggestion adds to training data for future model improvements. With this data in hand, it was inevitable that model quality would improve dramatically over time. Notably, other markets in this quadrant (e.g., slide generation) that don’t have the same fine-grained feedback loop have seen much slower improvements.

Anything in the “hard to solve” category is going to require significant investment – across token usage, technical talent, and likely eventually model training and RL. The ease of adoption is a powerful data acquisition flywheel that enables that deeper investment. The frontier model labs seem to view these kinds of widely used productivity agents as being in their domain. They’re already competing heavily on coding agents, and we would not be surprised to see them launch more office-suite productivity tools beyond the document editors they already have. In other words, our prediction is that these markets are going to have heavyweight fights – smaller players will struggle to compete without huge capital outlays.

Stickiness, however, continues to be low here. Many of us run multiple coding agents, and as office productivity tools improve, there’s no reason that you wouldn’t jump to whichever app makes you the prettiest slides. The argument for stickiness is company-specific customization (e.g., Cursor rules, brand templates), but it’s possible we will see interoperability or a single standard emerge to enable migration.

Hard to adopt, easy to solve

This is the area where enterprise adoption of AI has really taken off in the last two years. When we say easy to solve, we’re not implying that there’s no product depth, but it’s easy to imagine how an LLM can execute a playbook for an e-commerce return or a password reset. Given that most enterprises are looking for wins from AI, the “obvious” problems are where they’ve turned for immediate adoption. That’s enabled an incredible pace of revenue growth for the leaders in these markets.

Two key things differentiate this quadrant. First, these products are not individually adoptable – buying an agent to handle support tickets or IT helpdesk requests is an organization-level decision that likely has a buying committee. Second, the comparative simplicity of the use case is offset by the difficult and tedious reality of enterprise integrations. The teams that can navigate legacy enterprise systems have a huge leg up.

That integration story is where there’s a data moat. While the data you get from these agents is less broadly applicable – and enterprises will likely restrict your ability to train models with it – you’re gathering data about how each customer works. Over time, that will help you make your product at large better, but most importantly, your product will become stickier for each customer. The next agent that comes along will have a hard time recreating that learned expertise.

In this area, investors are treating the larger startups as de facto incumbents. That’s not to say that there isn’t product innovation left to be done – there very likely is! – but it’s not immediately obvious why a smaller startup would be able to compete with the likes of Sierra and Decagon, for example. What’s less clear is whether the capital these companies are raising is primarily being used to drive GTM or whether there is a clear technical moat that’s emerging, à la coding-specific models. If it’s only the former, then startups might have to resort to competing on cost.

Hard to adopt, hard to solve.

Example apps: SRE, security ops

Hard to adopt, hard to solve problems have received (comparatively) the least attention out of all four quadrants. The potential value of solving complex engineering or operations workflows can be incredibly high, as these are tasks that typically take humans hours or days. Unfortunately, these are workflows that are also fairly custom on a company-by-company basis, which means evaluation and implementation are much more cumbersome than “easy to solve, hard to adopt” products.

We’ve placed our bet in the hard-hard quadrant, and this is where we expect to see the next phase of growth. The hard-hard markets will grow very quickly in the next couple years for a handful of reasons. First, reasoning models are now capable of planning to handle more complex tasks, which will help grapple with multi-step solutions. Second, a lot of the complexity in solving these problems comes from the steps outside of AI – building and configuring workflows; that will get easier and faster as coding agents get better. Finally, enterprises are already actively plucking the low-hanging fruit and will look to harder problems once those run out.

The data moat here is the most complex and potentially the most valuable. If you build expertise in one company’s workflows, that becomes very difficult to replicate – switching products would be akin to firing an experienced engineer and replacing them with a new person. There’s potentially an opportunity to build expertise in core capabilities (e.g., an SRE agent that’s an expert in AWS). However, this improvement cycle will be significantly slower than it was with coding agents because the quantity of data is lower and verifiability is less obvious.

While every one of these markets has a company that has raised astronomical amounts of money (often well ahead of revenue growth), we have a hard time imagining that these companies are as entrenched as their equivalents in the “easy to solve, hard to adopt” category. There’s a very long game left to be played in this market.

Wrapping up

This map isn’t set in stone; both boundaries will change. On the complexity front, we’ve seen dramatic improvements in model capabilities every few months. However, model improvements seem to be plateauing, so there’s less interest on this axis.

The real excitement is around UX. We’ve long believed that the UX aspects of AI applications are underexplored. We would not be surprised to see new UX paradigms developed that change the way users adopt products. Claude Code on the web is probably the best recent example of this – by making a coding agent available in a web browser available to everyone, it’s allowed users who might be scared away by an IDE or terminal to access these tools.

Regardless of what path they take, our bet is that the next 12-24 months will see the rise of winners in the hard-hard quadrant. It won’t look as seamless as the growth of Sierra and Decagon has been – there will be longer evaluation cycles, more complex implementations, and likely an overall lower success rate. But as companies improve their process and data enables improved models, this is where incredible amounts of revenue can be generated.

We’ll be off next week for a summer break and back the week after!

Last fall, we wrote that OpenAI is too cheap to beat. To date, that’s still our most popular post with over 30k views on Substack. With a title like that, it generated the amount of strong opinions you’d generally expect — both agreeing and disagreeing with us. The general gist of that post is that the cost performance tradeoff that OpenAI was offering at the time was as close as to optimal as you were going to get.

Looking at the market today — about 6 months later — we have two distinct observations:

1. OpenAI still sells the best top-end models. Claude 3 Opus is the only model that’s achieved comparable Elo¹, and it’s about 3x as expensive as GPT-4 as of this writing. In that top class of model, GPT-4 is still your best cost performance tradeoff.

2. The gap between the top tier and second tier of models has shrunk dramatically. That tier is still primarily comprised of proprietary models (Sonnet, Haiku, and older versions of GPT-4), but critically, Llama 3 has squarely made its way into that second tier.

Another post we wrote last fall was about how open-source LLMs shouldn’t try to win; instead, we argued, they should serve as the bases for efficient fine-tunes for task-specific experts. There, we argued that these open models should get smaller and faster at their existing quality rather than getting bigger and better to try to compete with GPT-4.

Looking back, we got some things very right, and we got some things very wrong. At a high-level, the direction open LLMs are headed is incredibly promising. Looking at a breakdown of the LMSys Elo for the top models, you can see that the gaps have started to close.

Here’s where we think things stand today:

• Open LLMs are very, very good. Even just 3 months ago, we experimented with Mixtral 8x7B and Llama 2 and discarded both because their results weren’t good enough. Today, the quality we’re able to achieve with Llama 3 and Mixtral 8x22B, given their parameter size, is quite impressive. We’ve been increasing our experimentation with them recently and are considering replacing parts of our production inference stack with Llama 3.

• Open LLMs don’t need to get that much smaller. What we were really arguing for last fall was that open LLMs needed to get more efficient at the inference step. What we didn’t account for was that the inference process itself would get dramatically more efficient in the last 6 months — which is embarrassing given that we’re systems people! Today, you can get at least 10 tokens/s on a MacBook for Llama 3 and production-level inference on cloud deployments.

• This makes fine-tuning open LLMs incredibly attractive. With the increasing quality of models and the plummeting costs (plus the fact that GPU availability is increasing quickly), fine-tuning open models becomes very attractive once more. There are obviously some details to work out across different model providers and the features they support. But as a team that relies on fine-tuned LLMs as a core part of our product, we can’t wait to find time to experiment with replacing fine-tuned GPT-3.5 with fine-tuned Llama 3. Our bet is that it will be higher quality, faster, and cheaper.

• But open models still won’t catch OpenAI. All that said, we still believe that open LLMs aren’t going to catch OpenAI. The scale advantage the proprietary model builders have — combined with the positive consumer feedback loop — is daunting. If Meta can’t get close with all its resources, it’s unlikely anyone else will — but that’s okay! Open LLMs don’t need to be the best models around to survive.

We’ve gone back-and-forth many times over the last year about whether RAG or fine-tuning will win. The answer today squarely seem to be “both” — but the path forward for efficient, scalable fine-tuning was always murky². That’s started to change in the last month, and that’s incredibly exciting. As we work out the details in fine-tuning Llama 3, we’ll very reasonably start to see a proliferation of these narrow, expert LLMs out in the world.

At Herald (recently rebranded from RunLLM, in case you haven’t seen), we’ve flipped our distribution model in the past couple months from a top-down, enterprise-oriented approach to a bottoms-up approach that enables developers to try our product quickly and with minimal change. To do that, we’ve rolled out a new CLI that’s up and running on your laptop in minutes – like Claude Code or Cursor – without requiring you to upload any credentials to the cloud.

We’d obviously love for you to install our CLI, try it out, and share any feedback that you have – but we also think that what we’ve learned reflects something fundamental about distribution of AI products.

What we learned

It’s worth starting with why we were previously in a different mode of distribution. Our product – an AI DevOps Agent – understands a team’s product, infrastructure, and business and helps engineers answer questions, debug issues, and discover incidents before customers complain. By its nature, this kind of agent requires access to a lot of different data: code, telemetry, infrastructure, CI/CD, documentation, and so on. To show off the full value of the agent, we encouraged our customers to connect as much data as possible – but for understandable reasons, engineering teams won’t blindly connect all of those data sources to a new agent without the appropriate security and legal review. The process we set up was to engage with a company’s leadership, get buy-in, go through legal & security review, get access to all the data, then show teams what we could do.

That works to an extent, but it leaves a lot of opportunity on the table. While getting the necessary approvals, we found that a number of teams – especially the most AI-enabled ones – would quickly move on to other priorities or go with an in-house solution. We could talk about why DIY solutions are worse until we’re blue in the face – most vendors are right now! – but the fact that teams can make themselves successful more quickly than ever is incontrovertible.

That’s what led us to conclude that we had to show engineers value immediately. Once they understood why we were better than what they could build in-house – even if with limited access – then we could justify the time and effort required to get comprehensive access. Of course, to do that, we had to address security concerns too. Long story short, we built a version of our agent that has a familiar form-factor (Claude Code-style CLI) but, critically, keeps all of your credentials local. An engineer can now install the agent with npm install -g @herald-ai/herald, point at the tools that they already use locally (e.g., AWS CLI, Grafana MCP, etc.), and see what Herald can do immediately.

The implications

None of this is to say that everyone should go build a CLI. We’re working with developers, so we set out to build something that was familiar to them, but your target audience, their ideal UX, and what value you can show in a limited capacity might very well be different. In many ways, building a CLI is actually harder than traditional product-led adoption because you lose the ability to instrument many of the adoption funnel signals that you can capture on a web app.

The traditional wisdom for a lot of developer-led adoption is that you want to show people the full value of the product up front rather than gating features. While that’s true, agents are all about data access, and an individual user or small team is never going to be able to provide full data access. Your product should be designed for that limitation. It’s reasonable if your product would be better with more data, but it needs to be good with limited data.

There’s unfortunately no one-size-fits-all solution to operating with limited data. In our case, we’ve customized an implementation of our predictive issue detection feature to each data source that a user might configure. The full implementation works better when there’s multiple data sources to correlate across – but the explicit compromise made it easier for us to show value to the user immediately. What compromise you can make to show value immediately is the question that should drive your implementation.

Given that we’re talking about an adoption strategy that puts product front and center, you might be tempted to think that we’re saying every agent should adopt a PLG motion. Not quite. While giving people a taste of what your agent can do up front matters, you very well might still need humans in the loop to solidify and grow usage. For our agent to truly be successful in an enterprise, we still need to go through security approvals and a team-wide deployment. Predictive issue detection will still be most valuable when the IT team connects all the relevant data sources. The difference is that we can now operate in both modes – limited data to show value quickly, and comprehensive data to show the full value.

Is this universal?

Our initial thought was that this was unique to developers – a lot of the people we talk to are Claude Code zealots, after all. The more we think about it and look around, the more we realize that this is a challenge that every software vendor is going to face in the next few years.

To be clear, there are exceptions. Companies like Sierra and Decagon seem to be growing at a breakneck pace with large enterprise contracts, and security products have always been bought & sold differently than other software infrastructure. It’s worth noting that these hard-to-adopt products have a unique data advantage that comes from embedding, and we’re explicitly advocating for a transition from hard- to easy-to-adopt – or at least a dual approach. That’s a topic for another post.

But again, these exceptions are noteworthy because plenty of software that you might expect to be purchased top-down – inference infrastructure, voice agents, sales tooling – is seeing a pattern where a usage-first adoption is driving growth. You should build for a world where every vendor has to show value now.

This line in particular stuck out to us:

Finance says, “half the budget,” engineering says, “double it” and you don’t know who’s right because there is no shared language of value. There’s no attribution, and no attribution means no allocation.

This got us thinking about what it actually means to allocate budget in AI and how enterprises should be thinking about it. What’s interesting about this challenge is that there has rarely been a truly universal resource in the enterprise. Marketing never asked for budget to run cloud infrastructure, and even if engineering built an internal tool for marketing, that was typically still considered engineering’s responsibility.

Platform plays are the closest we’ve historically gotten – plenty of companies will centralize on Salesforce, but from an invoicing perspective, you could very clearly break apart seat costs for Sales Cloud, Marketing Cloud, and Service Cloud. Now, you get a single invoice at the end of the month that shows a large number of tokens used. Who used them, to what end, and was there an ROI? Who the hell knows!

We have genuine sympathy for finance teams who are facing these challenges for the first time – and for the engineering leaders who are getting their “AI budgets” blown up because, well, everything is AI now. And that’s the most interesting thing to realize: Everything is AI now. There can’t be a single line item for AI because AI is in every part of your budget. So what do you do instead?

The funny thing is that it’s not all that interesting. AI budgeting is a solved problem – the industry collectively learned how to account for variable costs from cloud infrastructure, and budgeting for AI spend is the same. What needs to catch up is the infrastructure to make this happen.

Don’t confuse applications and intelligence

This might sound trivially obvious, but it seems to get muddled more often than we might think. It’s the most obvious place to start. The X discourse is all about tokens for obvious reasons: Software is no longer zero-marginal cost, and that means that your bills genuinely are climbing faster than you’d realize with the use of AI. But your large monthly Anthropic invoice is different from an agent that’s built for a particular application area. Spend on verticalized agents should obviously go into the budget for the specific organization that is using that agent, and the team that’s buying that agent – just like with any other software! – should be precise and empirical with evaluating what the potential ROI is. You’d be shocked how often we talk to customers who seem to have given almost no thought to how they would measure the success of the agent. If there isn’t an ROI, there isn’t going to be a budget.

The interesting thing here is that application spend is getting harder to predict. Most agents are adopting token-based billing (or some abstraction on top of tokens) with some occasional seat-based access layered on top. For most of us, the token cost dominates the seat cost by at least an order of magnitude. How do you budget when your spend can vary so much? Again, history gives us clear answers. Lest we forget, this was the exact concern with (and criticism of) serverless computing ~8-10 years ago: There was no way for enterprises to know how much a workload would cost. But companies found a way around this challenge, allocating a bucket of spend to cloud providers and then burning down that spend through their own usage or through marketplace purchases. The same model will very likely emerge for AI spend.

Budgeting for “raw” intelligence is a much harder problem, and that’s where the infrastructure needs to catch up.

Engineering discipline is enterprise discipline

Engineering teams have historically had to develop good abstractions for resource management, and the rest of the enterprise is going to have to catch up. Developers don’t run tests on the production cluster, and the staging and production databases have to be isolated. That makes it dramatically easier for a finance team to understand COGS, development spend, and potential sources of waste.

That same level of visibility is going to need to be adopted to meter AI spend accordingly. Today, tools like Claude and Cursor give you visibility into how many tokens each user used, but that’s just the starting point. You have no idea if those tokens were spent on useful work, new experiments, or totally useless things. On one hand, you don’t want to be metering so closely that you discourage use – it’s fine if someone spends a couple bucks looking up where to eat lunch today – but visibility matters.

This is where products like Entire are particularly interesting to us. Creating a clear audit trail to help teams understand what work was done, which agents were responsible for it, and how that ties back to the ultimate goals that the company sets is a critical component in understanding what AI spend is actually worth – and where the waste is happening. While it’s very early, we have a hunch that spend management and cost optimization might end up being an underrated application area for tools like Entire.

The interesting question, though, is how that same discipline extends into other areas. Again, engineering by its nature is structured. Coding agent spend can be tracked as a part of a git commit. Some areas – e.g., customer support – have natural analogs, and you can easily imagine tracking how much was spent in resolving a particular ticket. Others like marketing and sales are dramatically less amenable to precise tracking in their current structure. While we’re not experts in exactly how you should empiricize those functional areas, we think it’s somewhat inevitable that good visibility and structure on token spend will become integral to how those areas are run.

Get with the times

None of this is going to be tidy in the interim. Teams will experiment, budgets will be disorganized, and spend will run over what you expect. That’s fine – that’s what the early years of cloud spend looked like too! The mistake is responding to that fuzziness by painting with a broad brush. You absolutely should not be sticking all of your AI spend into one budget and saddling the engineering team with it. That might feel like an okay short-term fix, but it’ll hold your whole company back.

None of this requires inventing a new discipline. The accounting model already exists and can be copied from cloud infrastructure. What’s missing is the visibily layer, and that’s being built right now. Eric Glyman is right that no attribution means no allocation, but attribution is an engineering problem – and engineering problems get solved incredibly fast nowadays. The finance teams that handle this transition well will be the ones who recognize that AI should be treated like the next phase of infrastructure.

Headlines over the last few weeks have been filled with discussion of companies like Uber dramatically running over their AI token budgets for the year, unexpected Claude overages, and whether you can measure the impact from coding agents. And of course, there’s the backlash to the backlash. We’ve heard stories ourselves about CTOs and VPs of Engineering freezing budgets and instituting extreme scrutiny on all new AI spend.

Token leaderboards are no more intelligent than ranking engineers based on lines of code generated. Even if you assume that everyone was behaving with the best of intentions, using more tokens doesn’t mean you got more done. You’re measuring the wrong thing and allocating your budget the wrong way.

What happens next is somewhat predictable. Enterprises will pull back on token spend, institute per-employee limits, and have teams submit budget requests for token allocations. Budget freezes will be widespread practice by the end of the year. From our perspective, this is completely the wrong way to approach AI spend. You shouldn’t be turning up spend to brag about how big the number is, and you shouldn’t be limiting AI use out of a fear of overages because you were previously tokenmaxxing. In fact, thinking about your AI spend as one number is the wrong way to look at it.

Different agents – and different experiments with new tools – are going to have different impact on your business. It makes no more sense to have one number for all of your AI spend than it does to have one number for all of your salary spend. Instead, you should think about each AI tool as impacting the team or the functionality buying it. When put in that frame, the rational rules for AI spend start to look pretty different. We have a few rules that we’ve started to solidify both from our own experiences and from talking to hundreds of engineering organizations.

Limits with limits

Enterprises need budgets, and we’re not advocating for unrestrained use or $500M Claude bills being the new norm. But you need to set limits on how you set limits. Going from no constraints on usage and massive bills to extreme scrutiny and justification for every expense both creates organizational whiplash (how often do I have to change my habits) and also discourages experimentation. Different uses of AI – proven value adds like coding agents vs. experimental new tools – should explicitly be treated differently. If you don’t have a budget for experimenting with new tools, you’re going to fall behind.

Discouraging experimentation is probably the biggest and most avoidable own-goal. While the pace of change is very high, there’s still tons of undiscovered or poorly understood application areas, and aggressively setting limits on budgets means that your team isn’t going to learn about what’s possible. We’ve already heard engineering teams share that they’re not being allowed to spend on any new AI tools even while they believe our solution would solve a real problem for them – and would cost 1-2 orders of magnitude less than what they’re spending on salaries solving the problem (or on OpenAI or Anthropic for that matter).

This also requires reframing your thinking on software vs. headcount budget. Traditionally, software budgets and salary budgets have been treated differently, but we’ve seen cases where teams have open headcount for which they can’t find high-quality talent, while they’re blocked from spending more money on software. Agents won’t replace humans 1-for-1, but they will defray the tedious work that allows people to focus on what they’re best at.

When you limit experimentation, you’re embracing a static worldview. You’re limiting your team’s ability to adapt to the latest technology, and relying only on headcount is the hardest way to solve capacity problems. You certainly don’t want to be blowing out your budgets, but giving your competitors an advantage by sitting on your hands and pretending the technology around you isn’t changing doesn’t work either.

Build vs. buy: The calculus (isn’t all that) different

There’s plenty of ink being spilled on whether you should build all your software moving forward. We’ve talked about the build vs. buy calculus before, so we won’t repeat the arguments in full detail, but the build vs. buy decision plays into the budget discussion. Building a product in-house is more likely to lead to budget overruns than buying something off the shelf. Humans are generally terrible at sunk costs, so once you invest into building a prototype, it’s natural to double down on that effort – but that’s where your budget overrun is most likely to get worse.

It’s a tired trope at this point that coding agents make it mind-numbingly easy to build a good-enough demo. It takes minutes and costs pennies. But the iteration and time that’s required to take that demo and turn it into a useful product is not quite so simple – or cheap. You’re both going to spend valuable time on that productionization process and you’re going to spend tons of tokens fixing all the bugs and edge cases that you didn’t anticipate in the demo that you built in just a few minutes. More importantly, when a coding agent writes most of the code, no one’s going to know what happened when it breaks. That means Claude is going to burn tons of tokens figuring out the issue.

This is where the budget argument and ROI calculus get particularly difficult. Is this engineering expense or operational cost? How much of this is expected, and how much will stability increase with time? How do you account for the time humans are spending on this problem as opposed to others? Properly estimating complexity and allocating budget for a piece of software that’s not your key expertise is inevitably going to be noisy and inaccurate – and that inaccuracy will spill over to affect the rest of your “AI budget.” The token cost will look tiny in the build phase – but when you get to productionization, maintenance, and new features, the token and salary math get very hard to justify.

Prioritization is more important than ever

We’re noticing that engineering teams are increasingly distracted – we shared an anecdote a couple weeks ago about an engineering team that was jumping between possible solutions. To some extent, this is understandable because engineering teams really can do anything right now. But running tons of experiments that end up resulting in very little (especially when there are off-the-shelf solutions available) is an expensive distraction. Once again, this runs up token costs – and the budget experiments that have dubious value shouldn’t be conflated with real systems that can help your team.

This is perhaps a more subtle challenge at first glance. Each potential experiment that you run that isn’t well thought out can go from a $5 starting point to a $1,000 sinkhole very quickly. We’ve had this happen ourselves. While we’re not advocating for shutting down all agent use for the purposes of keeping a lid on budgets, knowing what you’re trying to accomplish – and how that ties into your token use – is critical. Giving your team room to experiment is valuable, but that’s not the same as something that’s directly adding value to the business. Mixing the two up leads to bad decision making about “AI.”

AI is not a line item

The question underlying this whole conversation is what the budget for AI tools should be, which is (literally) a million-dollar question. We’ve seen a pretty wide range of approaches – no limits at all (quickly becoming a thing of the past), everything goes to engineering, or an AI budget per-organization. But in all cases, AI is being treated as a line item on the budget – the wrong approach.

This isn’t a question about whether AI is here to stay, but about how we think about the value AI is adding. The value AI is adding to shipping product features that directly affect your top line is very different from the value it’s adding when you’re experimenting with recreating a third-party tool – or from the value it adds in writing cold email copy. Treating that all as “AI spend” is misguided. Different agents are going to add different amounts of value and in different ways.

That also doesn’t mean that you should treat every AI agent as one that’s going to have immediately measurable ROI – you should absolutely have budget allocated to experiment with things that might add value (or might not). How large that number is and how strictly it’s metered probably varies from business to business. Either way, putting all AI spend in one bucket and tokenmaxxing that number up or hurriedly rushing it down is a recipe for disaster.

We later confirmed with the team that we weren’t crazy. We knew more about the state of his team’s operations than he did.

This is not particularly new for anyone who’s worked at or sold to big companies before. There are plenty of leaders who aren’t connected to what their team is actually doing – enterprises have long succeeded and grown despite this disconnect. Unfortunately, the perils of this disconnect have gotten orders of magnitude more pronounced with AI.

Leaders who don’t understand their organizations are going to prioritize initiatives and buy agents that increase disorganization without driving results. Here’s why: Agents drive real value when they conform to enterprises’ workflows, which means you need a clear understanding of what your team does today and what’s needed to improve. Without that understanding, leaders will thrash between possible solutions, and in the worst case, they will end up purchasing solutions that amplify existing issues. That means agents should be implemented where there’s a clear picture of the current state of play as well as a deep understanding of what good looks like. If you don’t know what good is, an LLM isn’t going to tell you.

Focus still matters

Historically, engineering teams have had to focus aggressively before resources were scarce. There’s always been a million things that could be worked on – niche features, tech debt, performance optimization, internal tools, etc. – but cycles to only work on a few of them. To some extent, coding agents have alleviated that pressure, and teams can now spin up new tools (and certainly flashy demos!) faster than before.

What happens after that still requires strong focus. Every prototype can’t be taken to production, because the process of refining and maintaining that prototype paradoxically has a much higher opportunity cost than before. It is definitely true that engineering teams are faster than before, but that means time spent on productionizing something that could be bought would be better spent improving core functionality that’s in the business’ wheelhouse.

This is where the disconnect happened in our conversation last week: The VP was anchored on a demo he’d been shown that relied on hard-coded workflows and runbooks and wouldn’t generalize to the complexity of their stack. On top of that, he also wasn’t aware about the manual workflows the team currently followed, which would make the demo even more difficult to generalize.

To be clear, the VP wasn’t gratuitously trying to tear us down or being blithely cynical. He was operating the way that senior leaders at large enterprises have typically operated for decades. But with coding agents and flashy demos, it’s easier than ever to convince yourself that the world looks dramatically different than it does in reality.

What happens when you’re misled? You make decisions about what you think needs to be done rather than what actually needs to be done. We started a conversation last summer with a mid-sized tech company that wanted to use an AI SRE to deal with thousands of PagerDuty alerts a week. They evaluated us and one other vendor, but by the end of the year hadn’t yet come to a decision because the CEO suggested they do two more evaluations. Those evaluations last until the end of Q1, and by that point, an engineer had built an in-house solution the CEO got excited about because it dogfooded their internal product. Eight months of evaluation time was wasted, and now there are two engineers on the hook for maintaining a side project.

In each of these cases, an unrealistic ideal became a blocker for a good solution.

The LLM has no clothes

A lack of clarity means that your agent implementations lack a clear target for what’s actually being improved. In many cases, you can very much accelerate speed with agents – but speed without quality mostly just creates noise.

Take coding as an example. Adopting coding agents certainly means that you’ll have cleaner syntax and better comments. Your engineers might even prompt the agent to write more tests! That’s not going to make better software in general. Coding agents aren’t going to magically give you high-quality design, thoughtful system architecture, or a culture that prioritizes catching bugs before they ship. If your team is in the habit of not ensuring their implementations closely match product requirements and validating that edge cases are properly handled, that problem is likely going to be amplified with AI.

One of the first POCs we did in the AI SRE space was with a traditional software vendor – the POC ended up failing. The organization was excited about AI as a strategic initiative for them, but their stack simply was not set up for integration with agents – their existing debugging workflows were full of manual processes that required humans to use credentials with root access to access individual deployments and pull the relevant data. They were understandably hesitant about giving an agent this kind of information, but that meant the agent was operating with limited context – and ultimately, its recommendations were useless. Their processes were immature, and the agent created more noise.

The lesson is simple: Know what good looks like before you adopt an agent. If not, you’ll be impressed by the shiny new clothes that turn out not to be real.

Adapting to reality

This changes the way that software is evaluated and purchased in a few key ways. Teams building agents must be flexible. As we talked about last week, every AI startup needs to be oriented towards showing value quickly and orienting pricing and commercial terms to quick adoption. If you’re building a genuinely good product, that’s how you’ll avoid leading your customers into the traps that we discussed above. Trust us, we’ve made these mistakes!

There are also clear lessons for buyers. First, it requires trusting your team. As a leader, you need to enable your team to make the call about what agents are worth spending time on and what will deliver the results that align with your priorities. The responsibility to make decisions about which agents are good or bad has to rest with the people doing the work.

Second, there will be trial and error – and you’ll have to get used to it. We’ve heard anecdotes about major law firms buying all three of the top AI legal agents and letting their organizations use them in parallel with the intention of picking one after a year or two. While this might sound wasteful, it’s a wiser approach than being beholden to one implementation up front. It’s better to learn now rather than a year from now.

Finally, we find the idea of flat organizations particularly interesting. Brian Chesky talked about this in a recent interview: The closer you can get to the ground truth – the actual work being done and the problems being faced – the more likely you are to make intelligent decisions about what’s working and what’s not. The more abstraction you have between yourself and reality, the more likely you are to think that your team is light years ahead or behind where it actually is.

Ultimately, focus is what will win. AI that lives at the intersection of possible and good is what will create value.

Enterprise AI is in a critical transition and maturation period – we’re calling these the teenage years. From 2023-2025, we were the “kid in a toy store phase” – budgets were unconstrained, and most shiny demos could find an excited buyer. The teenage years are going to be awkward and uncertain and that means that traditional enterprise sales patterns don’t work. This phase won’t last — the market will eventually settle back into something resembling traditional enterprise software — but the vendors who navigate the window well will be the ones still standing when it closes. That means giving up on the old playbook and meeting customers where they actually are.

What enterprises are actually doing

Three things define how enterprises are buying AI right now, and none of them line up with how the enterprise sales motion was designed to work.

The first is genuine uncertainty about the future, and it’s the most obviously understandable of the three. The last twelve months of model and tooling progress have made any multi-year commitment feel risky. Buyers are looking at what happened when Claude Code and Codex took a step-function leap last fall, and they’re correctly inferring that there very well might be a next equivalent sea change in six or twelve months. Locking into a three-year contract with any vendor right now feels like signing a lease on a house you might want to leave before the paint dries.

The second is a sharply increased appetite for building in-house. The quality of coding agents has shifted the build-vs-buy calculus so dramatically that “let’s also throw together an internal version and see how it compares” has become a standard step in evaluations. We’re doing this ourselves: instead of keeping a web design consultancy on retainer, we recently rebuilt our entire website from scratch and wired it into a headless CMS, and the whole thing cost us less than $100 in tokens. The math that used to make in-house an obvious act of engineering hubris is no longer obvious. Of course, this doesn’t take into account the ongoing maintenance and the fact that same DIY effort could be applied to 10 other initiatives.

The third is genuinely chaotic decision-making. VPs who missed the last big trend – cloud, mobile, etc. – don’t want to miss AI. The customer we mentioned at the top isn’t unusual. We regularly see deals where buyers are simultaneously evaluating multiple vendors, building something themselves, and migrating an underlying tool that affects the whole evaluation — none of which they bring up until something forces it into the conversation. The speed of change has compressed everyone’s planning horizon to the point that sequential decision-making has stopped working.

Traditional playbooks are wrong

If you’ve sold enterprise software before, your instincts are wrong.

The default response to a build-curious customer is to argue them out of it. You walk them through the maintenance burden, the opportunity-cost of engineering cycles, and the advanced features they haven’t considered. Normally, this is pretty compelling. Today, it almost guarantees you lose the deal. Building something yourself is fun and exciting and feels like an exercise of agency. The harder you push against it, the more it seems like you’re a vendor making a desperate argument.

You have two options here, and you have to pick the one that works best for you. The first option is to build something that adds value faster than the DIY solution takes shape – your product has to be configured and onboarded immediately. Anything longer than a weekend loses to DIY. The hackathon project isn’t going to have the same quality, but if your alternative is a 6-month enterprise sales cycle, a half-good solution today is better. By the time you’ve finished your POC, the customer has either built their own muscle around the homegrown version or moved on to whatever the next exciting thing is.

The second option is to become an enabler for the DIY engineers: Let them build an interface on top of your expertise, so they both get a sense of agency from building and move faster than they would have otherwise. This isn’t necessarily something every company can do, but enabling DIY-curious engineers can be very powerful.

The second instinct that’s wrong is pushing for longer commits and more structured POCs. Traditional enterprise selling rewards discipline here: show value, propose a multi-year deal, get the procurement process running, and get the buyer locked in. In the current market, the opposite is winning. The best AI companies we see are offering zero or minimal commitment upfront, getting customers onboarded fast, and growing contracts over time as value becomes obvious. This gives the vendor a quicker win while giving the buyer the flexibility in case the world changes again in a few months. An interesting paradox we’ve seen is that the flexibility to leave sometimes produces more investment in making the product successful – investment feels safe because optionality exists.

Finally, there’s the chaos of enterprise decision-making. Your risk is again in timing – time kills all deals, and the solution is, again, flexibility. Land quickly and efficiently, enable your product to become entrenched, and grow from there.

From teenage to adulthood

The transitional phase isn’t permanent, and the playbook that wins it isn’t the playbook you’ll need for the next phase.

On pricing, the current vogue for usage-based and outcome-based pricing is going to run into the wall of enterprise quarterly planning. Aligning what you charge for with the unit of actual work done is conceptually clean — it’s how consulting contracts have always worked — but enterprises are going to struggle to reconcile unpredictable monthly bills with quarterly forecasts. We’re already hearing about public companies having end-of-quarter panics over engineering budgets because spend ran wildly over. The likely endpoint is a return to longer-term, more fixed commitments — but with more sophistication about right-sizing commitments. The short-term will stay messy, because enterprises are correctly prioritizing flexibility now. The vendors who win the transition will be the ones who use the flexibility window to get embedded, then transition into more predictable commercial structures.

On the build-vs-buy front, we’ll likely see the euphoria tamped down. In the next year or two, an enterprise is going to make headlines for trying to homegrow a CRM, ERP, or payroll system, and it’s going to backfire spectacularly. The lesson won’t be “stop building”; it’ll be “build the things that are genuinely different for your business, and buy everything else.” In other words, the conventional wisdom will be true again.

Longer term, we think the market will eventually head back toward something that looks like traditional enterprise software — well-understood buying cycles, comparable feature matrices, predictable evaluation processes. The difference is that the companies running those cycles two years from now will be the ones who survived the in-between by abandoning the old playbook.

Hello fellow kids

The temptation in a market this strange is to keep trying to make it behave the way the old market behaved – or to throw caution to the wind and do something totally different. Reality is as always somewhere in between. To push for a formal sales cycle or for the long commit doesn’t work right now. At the same time, we aren’t in an alternate dimension where the laws of physics are backwards. Enterprise software vendors like Notion, Vercel, and Datadog figured out how to deliver value to companies quickly over the past decade. That trend has spread everywhere today. What you ultimately need to understand is what your customers’ incentives are – those genuinely are different from 5 years ago – and how you can best enable them.

The transition phase is going to keep being chaotic for a while. The after is going to look a lot more like the traditional enterprise software market we all know. The window between the two is open right now, and the vendors who use it well will be the ones still standing when it closes.

We’ve written a few times before about how AI is breaking the playbooks that software companies have relied on for decades. Most of those posts have been about the way the technology itself shifts what’s possible — commoditization of models, data as the only real moat, the gap between enthusiasts and the rest of the market. We’ve begun to suspect that the changes might be even more fundamental: Even if you do everything right, the act of operationalizing what’s working is now the thing that puts you most at risk.

In AI, product-market fit is a trap. Not because the thing you found doesn’t work — it almost certainly does — but because the discipline of turning it into a repeatable machine is what makes you miss the next thing. And the next thing arrives every few weeks.

Operationalizing kills your signal

The traditional advice once a company finds product-market fit is to operationalize relentlessly. On the sales side, you codify your ICP, hire a large sales team, write playbooks, and instrument the funnel. On the product side, you deepen the functionality that got you to this point. The whole goal of post-product-market fit life is to systematize what you suspect works. Historically, this is what separated companies that scale from companies that have a good first year and then plateau.

The goal of operationalizing a process is to create consistency: As the saying goes, investors want to know that putting $N into sales & marketing is going to yield $M in net-new revenue. It just might be the case that consistency – which is the holy grail of a growing business – is also a risk in AI.

At the early stages of creating a business, you might have a particularly interesting customer call, come up with a new idea, and change your product roadmap for the next two months. When you’re running a machine that’s meant to scale, you can’t turn on a dime. The concern is that your customers’ preferences are changing faster than ever before – the cool demo they saw on Twitter last night might be their new benchmark for what good in your market is. If that signal takes weeks to spread across your sales team and work back to the product team, you might already be too late.

Cursor’s situation is a useful illustration here. They were able to scale revenue ridiculously quickly in 2024 and 2025, and they raised massive amounts of capital to deepen their moat and scale. Over the last 6 months, as Claude Code and Codex have turned into extremely formidable competition, the sheen has worn off. We don’t want to overstate the point – a potentially $60B acquisition by SpaceX would still be an incredible outcome! – but anecdotally, Cursor is not the it tool that it was a year ago. A combination of the UX improvements in Claude and Codex and the increased value in vertical integration between coding agent and model provider are the current advantages. The exact reason Cursor was so successful 12 months ago – being a sleek, minimal IDE that was model agnostic – might be what’s dragging them down today. Of course, things might flip next week!

This is a Catch-22. If we saw the growth that Cursor did in 2024, we too would have made the same decisions. If they didn’t, perhaps someone else would have and grown faster than them, and they would be lost to history. What we’re pretty sure about is that – more than ever – you constantly have to be replacing your own product.

The early-late paradox

The second reason PMF is now a trap is more structural, and it’s the part of this we’re least sure how to solve.

Crossing the Chasm canonically outlined that you sold to early adopters first, used what you learned to build a product the early majority would tolerate, and then over years made your way through the late majority and the laggards. The model assumed that these segments were sequential and that they roughly corresponded to different companies. You went after the tech-forward shops first, then the mainstream, then the slow-movers. The product, the sales motion, and the messaging could evolve in step with the segment you were chasing.

That model doesn’t really describe what we see anymore. The early adopters and the late majority are now sitting at the same company — sometimes on the same team. We have one call with an engineer who has rebuilt their entire workflow around Claude Code and is impatient for the next leap; the very next call is with someone, two desks over, who tried a coding agent once, didn’t like the UX, and quietly went back to writing things by hand.

We discussed this in detail in AI companies are building for the wrong users – an excited user might look past the flaws in your UX and ask for innovative new features while the reluctant user might stop paying attention to you at the first sign of a clumsy implementation. Neither one is wrong necessarily, but building each kind of product requires a different muscle.

You can’t build two products, because you’re a startup and you have to ship something. You can’t split the difference, because a product designed to be acceptable to both ends up being remarkable to neither. The honest answer is that you have to pick — and once you pick, you have to commit, and hope that the segment you bet on is the one that pulls the rest of the organization along.

This is what makes the operationalization problem from the previous section especially acute. The natural instinct, once you find a buyer who loves you, is to build the GTM machine around that buyer. But if that buyer is the early adopter inside an organization where the late majority controls the rollout, the machine you’ve built is selling to the wrong half of the room. And the more efficient that machine gets, the harder it becomes to even notice that the other half exists.

Are we all out of luck?

Most of the writing about AI strategy ends with some version of “stay flexible,” which is true but not actionable. The more honest version of the conclusion is this: in this market, the companies that win are not going to be the ones that find product-market fit first. In fact, we would even argue that systematizing first might be a disadvantage because it locks you into a bet while the market is still developing.

That sounds like heresy if you grew up on the SaaS playbook. But the SaaS playbook was built for a market where the workflow you sold to in year one was approximately the same workflow your customer would have in year four. In AI, that is no longer true. The playbook might not last a quarter because customers’ preferences are being rebuilt every week.

There’s a lot of chatter about rethinking how organizations are built in the age of AI – to be leaner, to be flatter, and to have more ownership. This is perhaps the most compelling reason why: Information about what customers are saying, where they’re failing, and what you need to do differently must travel faster than before. Product roadmaps can no longer be quarters or years out, and GTM motions need to change just as fast.

The good news is that we believe this will lead to more winners, not fewer. Every previous platform shift produced a handful of dominant companies who locked in their position early and held it for a decade or more. Whatever we think works today might change tomorrow and swap back the next day. Being ahead isn’t as safe as you think because you’re doing what worked last quarter. Iif you’re behind, you can still catch up.

This is a category error, and it’s distorting how everyone in AI reads the market. Cursor and Sierra are growing fast for very different reasons — and crucially, neither of those reasons applies to the hardest and most valuable problems AI is being pointed at. The infrastructure markets that look slow by comparison are not behind. They are operating on fundamentally different physics, and in many cases, the slowness is the thing that will make the eventual companies valuable.

The market map, revisited

We mapped this out earlier this year in Data is your only moat. The 2x2 we introduced in that post separates AI markets along two axes: how hard the problem is to solve technically, and how hard the product is to adopt organizationally. Three of the four quadrants are seeing fast growth right now, and each for a different reason. Easy-to-adopt, easy-to-solve (easy-easy) markets — consumer chat, basic search replacement — are growing because there’s almost no friction in either direction. The model providers are dominating these and that’s mostly fine. Easy-to-adopt, hard-to-solve markets, the Cursor quadrant, are growing because individual users can adopt without organizational approval and the data flywheel that follows is incredibly powerful. The core AI capabilities are racing ahead, and it’s hard to catch up if you’re not already in the game.

The hard-to-adopt-easy-to-solve quadrant is the more interesting case for this discussion, because the growth there has been remarkable too. Sierra, Decagon, and other enterprise support agents are scaling rapidly despite needing committee buy-in to land each customer – an individual support rep can’t buy Sierra. The reason is that the underlying problem — execute a defined playbook for a known workflow — is tractable enough that vendors can demonstrate value quickly once they’re in the door. The large fundraises these companies have done signal credibility to a Fortune 500 buyer who needs to know you’ll still be around in three years.

That leaves the fourth quadrant: hard-to-adopt-hard-to-solve. SRE, security operations, complex infrastructure agents. This is the only quadrant that doesn’t have a Cursor or a Sierra growing into it at breakneck pace, and it’s where the misreading is doing the most damage. Founders in this quadrant get benchmarked against companies in the other three quadrants and asked why they’re not on the same curve. The honest answer is that they were never going to be — and more importantly, that the slowness is fundamental to the problems being solved here.

What slowness buys you

When a market doesn’t let you move at breakneck speed — because the buyer is a committee, because the integration is genuinely complex, because the customer needs weeks to evaluate — you are forced to spend that time figuring out what the right product actually is. In a fast market, you don’t get that time. The competitive pressure is too high, the customers are too willing to start immediately, and the only correct move is to ship the obvious solution as quickly as possible and iterate from there.

Certainly for hard problems, the obvious solution is rarely the best one. Take AI SRE, which is where we spend our time. Almost every startup in the space (and there are many!) is building an AI-powered root cause analysis agent that is triggered by incident management alerts and requires customer-maintained runbooks. It’s the obvious solution: humans use runbooks, so the agent should too. Except alert thresholds are noisy, no engineering team actually maintains their runbooks well, and the agent inherits all the gaps. By trying to chase the Cursor growth curve, the players in this space haven’t asked the more interesting question: how do you use agents to detect early warning signs of an incident, validate them, and figure out the root cause from that structure — before any threshold alert ever fires?

To be transparent, we didn’t see this initially either. When we started, we set out to build the same banal runbook-driven RCA agent everyone else was building. The reason we ended up somewhere different is that the market gave us time. If customers had been willing to write us $250K checks on day one for a generic RCA agent, we would have shipped that and spent the next two years executing on the wrong product. The slowness of the market is what pushed us to keep asking whether the obvious solution was the right one. It turned out it wasn’t.

This is the first-order benefit of a slow market: it forces you to solve the actual problem, not the one that’s easiest to monetize.

Understanding this dynamic also helps you tailor your approach to the market you’re in. When you try to grow too fast for the type of market you’re in, the pressure forces you to take POCs you shouldn’t take, ship to customers you can’t actually serve well, and chase logos you don’t have the product depth to make successful. The cost isn’t just churn. It’s reputational damage that extends to the entire category. We’ve heard plenty of stories about AI agents in our market and adjacent ones that were rushed into enterprise environments, failed in visible ways, and left buyers with the strong sense that this whole category of product doesn’t work yet. That impression is sticky. It poisons the well not just for the vendor that failed but for everyone behind them trying to sell into the same buyers.

Education as a moat

The other key benefit is the one we underestimated coming into this experience: the education process itself becomes a moat. Most customers in hard-hard markets don’t fully know what an agent in their space is supposed to do. They have intuitions and pain points — but the category isn’t well-defined in their heads yet. That sounds like a problem, and in a fast market it would be. In a slow market, it’s an opportunity.

Look at finance agents for closing books, a market which perhaps is a few steps ahead of AI SRE on the same curve. Walk into any controller’s office and you’ll find a long list of vendors all positioned around roughly the same pitch: We automate your close. Some are reconciling against templates, some are categorizing transactions, some are flagging variance. The buyer’s job is to figure out which of these is actually solving their problem and which is selling them a glorified macro.

A vendor with a real point of view – we’re not experts, but Digits seems interesting – can stand out from the noise. The idea behind digits isn’t to put a pretty UI on top of your existing, messy general ledger data: They’re rebuilding the concept of a general ledger from scratch in order to support quicker, more efficient books that are constantly kept up to date. You might not agree with the approach, but it expresses a clear point of view that’s cohesive – not just AI slapped on top of whatever you had before. A differentiated point of view earns trust that competitors with vague positioning can’t easily replicate.

You don’t get to do this in a fast market. By the time Cursor was teaching anyone what coding agents could do, every developer on Twitter had already figured it out themselves. The market did the education for them. In hard-to-adopt markets, you do the education yourself, and the customers who learn from you tend to remember who taught them.

How to read these markets

Taken together — time to find the right solution, protection from category-poisoning failures, and education-as-differentiation — and you start to see why the slow curve isn’t just a consolation prize. It’s the trajectory that builds companies competitors can’t easily copy in a quarter when the funding cycle turns. The fast curve produces companies that grow incredibly quickly and then have to fight off model providers, incumbents, and a long tail of well-funded copycats with similar products. The slow curve produces companies that, by the time they’re visible, have a product nobody else can replicate without going through the same multi-year process.

This has implications for how you interpret business in these markets. The signals worth tracking are whether a company has a non-obvious technical thesis, whether their POC win rate is improving over time, whether their existing customers are deepening usage rather than just renewing. These are the signals that actually predict which companies in hard-hard markets will still be standing in five years.

The companies winning the fast AI markets right now deserve their growth. We’re not arguing otherwise. What we are arguing is that the curve they’re on is not the only curve, and treating it as the benchmark for all of AI is going to lead to a lot of misallocated capital, misread pipelines, and eventually, a lot of surprise when the slow-market companies turn out to be the ones with the deepest moats.

The next phase of AI isn’t going to look like Cursor’s growth chart. For the most valuable problems, it isn’t supposed to.

Last week, we wrote about some of the trends in the inference economy: plateauing token costs, using the right model for the right task, managing different modes of compute, and the impacts on pricing. Our perspective for the purposes of that that post was derived from observing what was going on with LLM inference in general in recent months. Of course, we’re always thinking about how that affects our decision-making as application builders, and we touched on that briefly but mostly focused on what was actually happening to token costs.

Looking back on that post, it occurred to us that the underlying assumption was that the dynamics of token consumption — from the perspective of application builders — is changing as well. Those changes are likely what’s driving the data center buildouts in the first place and are driving the demand side of the equation, which is forcing supply to keep up. As a result, today’s post is about what’s driving token token demand and how you should think about managing your demand as a token consumer.

Changes in token demand

What do we mean when we say that there’s a change in the demand side of the equation? Simply put, we’re all using more tokens to process more data. The increased token volume is partially driven by increased usage, but that’s far from the whole story. Yes, of course AI applications have more usage, but what’s much more interesting — and challenging — is that we’re seeing a trend in increased token consumption per-request, not just an increase in the aggregate number of requests processed. The driver behind that is ultimately quality.

As we’ve said many times on this blog, getting the behavior you want out of LLMs is all about providing the right information at the right time to a model. If you have the wrong context, you’re going to get poor results. That means the million dollar question is how you find the right information.

Search was the first solution that we all turned to — first, vector search, then returning back to more traditional text search mechanisms. Very quickly, however, we all turned to having an LLM read an input and evaluate how relevant it was to the problem we were solving (”reranking”). Presciently, Vik Singh, now a CVP at Microsoft, said this to us over 2 years ago: “If LLMs were fast enough… why not use the LLM to do a much more advanced similarity search… I think that’s what people actually want.”

The LLM-preprocesses-data paradigm is pervasive in our systems today. At RunLLM, we pre-read data at ingestion time to organize it properly, we read the results of text + vector search to analyze their relevance to a question, we analyze logs and dashboards in real-time with LLMs, and so on. Each one of those tasks is done by a model call in isolation to understand whether that data should be used for future decision-making. Without the involvement of LLMs in these stages, it would be almost impossible for us to provide high-quality results to our customers. We have a joke internally at RunLLM: The solution to every problem in computer science is another layer of abstraction, and the solution to every problem in AI is another layer of LLM calls.

That means that median — and perhaps more importantly, p99 — token consumption (and therefore request costs) are going up very quickly. We’re all solving harder problems, which means we’re throwing more data into LLMs and ultimately consuming dramatically more tokens. In our minds, this is one of the key drivers of increased token demand.

Luckily for the data center builders, this trend is not going anywhere. We might get more efficient and cheaper inference (though we’re skeptical, as we talked about last week), but as LLMs get more integrated into every application and workflow, per-request token use is only going to go up, not down. As a single data point, we have tons of ideas for how we can throw more LLMs at some of the challenges we face in a single investigation at RunLLM, but we’re primarily limited by cost, latency, or evaluations at the moment.

If you’re going to inevitably use more tokens, it’s worth thinking about how to be as thoughtful as possible about those tokens.

How to manage your token demand

We’re pretty confident token demand is going up, and as we discussed last week, token costs are plateauing. Depending on how long it takes to build and power these new datacenters, that means we should all be thinking about how to manage our token usage, especially as models get better and more expensive. We’ve been experimenting with many of these techniques for a while now at RunLLM, so we thought we’d share some early lessons.

• Model size is your best friend. Not all models are created equal, and neither are all tasks. Throwing your largest model at every task will probably maximize quality, but it will burn through your budget faster than you can imagine. (We mistakenly spent $63 on a single investigation at RunLLM last month. 😱) There are plenty of things that we do — gating questions, filtering documents, synthesizing logs — that aren’t hard but just require processing data efficiently. For simple tasks, there’s really no reason to use a state-of-the-art-model — GPT-4.1 Mini (one of our current favorites) or a smaller open-source equivalent will get the job done just fine. Unfortunately, we don’t have a cut-and-dry rule for when to use what model. It’s more of an art than a science right now, but evaluation frameworks for specific tasks certainly will help guide you in the right direction.

• Be flexible with your providers. We’ve long believed that LLM inference is a race to the bottom. If models get better, the main question becomes who can give you that model for as cheap as possible — especially with open-weight models. However, we touched on the fact that switching model providers is harder than it used to be last week because model providers are making stronger assumptions, but tools like DSPy make prompt optimization easier than ever, which should alleviate some of that tension. While you might not want to be ready to switch between every model provider on the market (there are a lot!), it’s probably worth your time to be ready to use one of a few different providers — or even using features like batch mode within individual providers — when you have the opportunity. The biggest issue with this is actually security & compliance: More data subprocessors create more data exposure risks and make your vendor approvals that much harder. But if you’re in an area where this is less of a concern, keeping your options open is definitely an option to reduce costs.

• Do you need reasoning? Reasoning models use a lot more tokens than regular LLMs, and it’s correspondingly quite difficult to control the output costs. It’s worth asking whether and when you need a reasoning model. For daily personal use, we tend to default to ChatGPT 5 Thinking, but we actually are not currently using any reasoning models in production at RunLLM. We’ve had much better luck with breaking problems down into fine-grained steps, using regular Python for orchestration and tool calling, and picking the right model for the right task (see above). Interestingly, this mirrors some of the reasoning-based task planning workflows we see in our daily usage, but with much stronger guardrails. Of course, we’re not solving problems with the generality of a consumer app like ChatGPT, so we have narrower score and much stronger guardrails. But for many workflow/task automation-oriented applications, you might be able to be much more efficient than you realize.

• Don’t run straight to fine-tuning/post-training. RL is a hot topic right now. As we mentioned last week, Cursor’s new custom autocomplete model has rekindled the excitement around fine-tuning models for custom tasks. It’s certainly appealing: You take a smaller model, feed it a bunch of data, and voilà — cheaper, faster inference. Unfortunately, reality is not quite that simple. For one thing, RL and post-training are hard. The promise of the recent wave of RL environment startups for post-training is that they will help remove the complexity here, but we’re not convinced that that’s a viable solution. The hard part isn’t running an algorithm to update weights — it’s framing the problem in a way that’s actually going to yield the results you want and getting enough data that fits that problem framing. (This is not a new challenge in RL.) What’s lost in the hype around Cursor is that they collected an immense amount of data well-suited to RL from the natural use of the product — every tab complete suggestion was accepted or rejected, which is a very friendly RL framing. By way of contrast, we have over 1MM question-answer pairs from RunLLM but a tiny fraction of those have actual feedback, and only a fraction of those have actionable feedback — e.g., we tend to get a most of our negative feedback on “I don’t know” answers, which are usually good because there’s not sufficient data to answer. If you’re in a domain where you have enough data and the expertise + resources to use post-training, it’s certainly viable from a unit margins perspective. But it’s not the panacea everyone’s claiming.

What’s interesting about these dynamics at the moment is that we’re all focused on costs but not as focused on our own pricing power. Of course, any business is always going to want to reduce its COGS — the more efficient you can be, the better your business scales. This is probably the right place to be given the fierce competitive dynamics in many AI markets. At the same time, while we’re working on technical solutions to reducing COGS, we’re also mindful of the fact that as AI applications mature and the ROI becomes more obvious, we’re likely going to see a corresponding increase in pricing power. The best applications will possibly command a significant premium. That won’t apply in every market of course — only in the ones where quality matters most.

Guesses aside, it’s clear that the economics of AI are changing faster than we would have expected. The sudden deceleration of per-token cost changes has coincided with more mature applications that require more tokens — a double-whammy to increase costs. There will be other solutions (technical and non-technical) that will change these dynamics, but for the foreseeable future, we’re all going to be keeping a close eye on our OpenAI bills.

Before we dive into the consequences of this debate, we’ll take a moment to define our terms.

The Python camp uses code to express strict requirements for the steps an agent should take to accomplish a task. The Markdown camp uses English to express broad goals and constraints and lets the agent plan its way to the outcome. The tradeoffs are fairly straightforward. Code creates strong guardrails and reduces the chance that the agent’s plan goes off the rails. Markdown gives powerful models the freedom to explore, adapts flexibly across tools and models, but risks the agent doing something unexpected and undesirable.

Most of the debate treats this as a choice between two defensible positions. It isn’t. Both maximalist positions are, in fact, failure modes, and the reason is the same: Neither one is actually agent-native. Agents, like humans, are increasingly being given complex tasks, and that requires the flexibility to choose the right tool for the right task (or subtask). Code-maximalism forces agents to follow deterministic workflows and strips out the reasoning that makes them useful. Markdown-maximalism abdicates control and produces systems you can’t debug, correct, or improve. Picking a side is how you avoid the hard work of designing an agent.

We’re publishing this as part of the Agent Native series because these two approaches increasingly define how agent interactions get built — and because both maximalist versions end up in the same place we wrote about last week: copy-pasting what a human would do, just in different syntax.

What code-maximalism gets wrong

The code-maximalist pitch is reliability. You tell the agent exactly what to do in specific cases, surface errors when things break, and get tightly scoped results. Given that LLMs make mistakes, misunderstand intent, and generally do all sorts of weird things, this sounds appealing in theory. Enforce correctness at the code layer. Don’t trust the model to do the right thing.

We’re intimately familiar with where this can go wrong in the AI SRE space. Almost every vendor in our space tells customers they have to write runbooks. The product then encodes those runbooks as workflows and has the agent execute them in response to specific alerts. The results are trustworthy in the narrow sense: the agent does roughly what you expected. It’s also useless the moment an alert looks different from anything that’s come before or the underlying architecture changes. We started down this misguided path ourselves in the early days and quickly learned that it would rarely work in practice.

This approach fails to be agent-native in three ways. First, it copy-pastes what a human does. A human picks one hypothesis — the most likely based on experience — and runs it down. That works when the human is confident, but when the initial hypothesis is wrong, it creates a lot of wasted work. An agent doesn’t have to fall into that trap. It can evaluate multiple hypotheses in parallel, and some will be dead ends, but the chance it lands on the right answer goes up dramatically. That’s the architecture we’ve built RunLLM around, and it’s consistently how we see real incidents get resolved.

Second, the runbook approach gives humans no meaningful visibility. SREs don’t need to confirm that the agent executed Step 3 of the runbook. They need to know what the agent tried, what it ruled out, and why. A well-worn path automates some tedious work, but it doesn’t let the human trust or learn from the agent’s reasoning.

Third, encoded workflows don’t evolve – they lose the intelligence that agents promise. When the underlying system changes or requirements shift, every encoding has to change with it. There’s no way for the agent to take feedback, understand that the expected behavior has changed, and adapt on its own without someone going back into the harness.

What Markdown-maximalism gets wrong

The Markdown-maximalist is optimized for flexibility. Describe the goal, hand it to a capable model, let it figure things out. This is portable, expressive, and gets you something working quickly. Where creativity or open-ended problem-solving matters, it can be dramatically more useful than a fixed workflow.

The degenerate version of this is AI slide generation. We don’t know the exact architecture behind these tools, but from the outside they read as “let the LLM do everything” applications — one prompt in, a whole slide deck out. The failure mode is familiar to anyone who’s used one. Something is off. The layout is weird on slide 7, the chart doesn’t match the claim, the flow of the argument is scrambled. You want to say: “On slide 7, make the flow vertical instead of horizontal and move the chart to the bottom.” You usually can’t get this to work the way you expect. There’s no discrete layout logic to adjust, no separable step for chart placement, no addressable unit smaller than the whole generation. You re-prompt, get a new deck that’s wrong in a different way, and start over.

It would be easy to write this off as a strawman. Serious Markdown-maximalists aren’t arguing for one-shotting every single application. The sophisticated version of the position is skills.md plus a basic agent loop — rich context, thoughtful instructions, and a capable model reasoning its way through. Guide the agent through context, the argument goes, rather than constraining it with fine-grained LLM calls.

Complex applications expose the gap. When you’re grappling with reality, there are plenty of engineering decisions that still require strict constraints: Context management and summarization, model selection, cost management, and cross-agent coordination to name a few. In each one of these cases, the challenge is not trusting the model to reason intelligently. It is building the tooling and infrastructure that allows a thoughtful model to execute these tasks efficiently and reliably.

In production, this results in a code harness that manages context, routes between models, orchestrates sub-agents, and handles the predictable places where pure prompting breaks down. That ends up being a hybrid architecture with markdown doing the guidance work and code doing the structural work — which is exactly the position the debate was supposed to be between.

If you start with a Markdown-maximalist architecture, you’re probably going to end up building plenty of narrow, harness-like capabilities – content management, model routing, etc. – to enforce constraints whether you like it or not. The question is just whether you design those hooks intentionally or let the code component grow organically. You should be intentional about the design.

The hybrid isn’t a compromise

The teams building serious agents have, largely independently, landed in the same place: Markdown for intent and domain guidance, code for enforcement, tool execution, and anything that must not fail silently. Claude Code works this way. We built RunLLM this way.

It’s tempting to read this as an unopinionated compromise. That’s the wrong framing. The whole point of agents is that – unlike traditional software – they have an understanding of the problem to be solved and can use the right tools to get there. Code-maximalism compromises on the planning and Markdown-maximalism compromises on execution and learning.

The reason hybrid architectures are winning is because they’re the only architectures that support what agents are actually supposed to do. An agent needs reasoning flexibility to handle situations it hasn’t seen before, and it needs deterministic guardrails so humans can trust it and intervene when needed. Neither extreme gives you both, which means neither maximalist position gives you a truly flexible agent. It gives you either a workflow with aspirations or a wish with nothing to execute it.

The architectural work is figuring out, for each part of your system, which layer it belongs to. What needs to be expressed as intent and reasoned about? What needs to be enforced and checked? Where does the agent need creativity, and where does it need constraints? This is the hard part, and it’s the part that picking a side lets you avoid.

What agent-native actually requires

When you stop treating Python vs. Markdown as the debate, the architectural priorities come into focus. Can your agent evaluate multiple hypotheses in parallel, or does it march down one? Can a human see what the agent tried and why, or do they just get a final answer? Can the agent adapt when the underlying system changes, or does someone need to go edit the harness? Can a user correct the output at the level of granularity they care about, or is it all-or-nothing?

The maximalist debates are a symptom of an industry still thinking about agents as workflow automators — either very rigid ones, or very loose ones. The teams building agent-native products are past that argument, because they’ve figured out that the argument was never really about Python or Markdown. It was about whether you were willing to do the work to build something that actually behaves like an agent.

We’re not sure exactly how the system is doing it – probably a small LLM call or maybe a glorified regex. It turns out that the system was looking at a title like “VP of Construction Engineering” and deciding that it was close enough to “VP of Engineering.” The model wasn’t given enough context about what kind of engineering we cared about in our ICP. It saw two titles that shared words and pattern-matched them together. We had been emailing VPs of Construction Engineering, which — for a developer tools company — is absolutely insane. It’s safe to say that we weren’t very happy with these results.

A human with even a shred of context handles this trivially – if someone on your SDR team was reaching out to construction companies regularly, you would seriously wonder if they were qualified for the role. But the agent had been designed the way most agents are designed today: take a task a human does, write down the steps, and hand each one to an LLM, then chain the results together. This step looked something like “compare the title to the list of accepted titles” – but without any context. The result is a workflow that works when everything fits the expected pattern and falls apart the moment anything requires judgment or is mildly under-specified.

This is the dominant approach to building agents right now, and we think it’s a legitimate anti-pattern. Most agents are built to copy-paste human workflows into automated pipelines. That’s destined to fail, because the things that make humans good at a task — context, judgment, the ability to notice when something feels off — are exactly the things that get lost in translation. The best agents aren’t built to act like humans. They’re built around a clear understanding of what agents do well that humans don’t, and vice versa.

The interactive agent wins

The clearest illustration of this is the difference in approach between coding agents that work interactively and the ones that tried to present themselves as autonomous software engineers. Products like Cursor started with tab completion and narrow task agents. That put the burden on software engineers to break tasks into small chunks, check in on the agent’s progress, and verify results. The agent did the tedious, high-volume part of code generation; the human brought context, taste, and judgment. Even today, as coding agents have dramatically improved, one of Claude’s strengths is not trying to do too much – focus on doing what the user asked, pause, and ask for more input.

On the other end of the spectrum were products that presented themselves as AI-powered software engineers, like Cognition’s Devin. You’d assign a whole task up front, and the agent would figure out the plan, gather context, and execute. The models simply weren’t ready for that. Without sufficient specification, the agent would regularly run off the rails and produce work that didn’t accomplish what anyone wanted. The scope was too broad, the feedback loop was too slow, and the human had no good way to course-correct until the damage was already done.

There’s a reason that the products that won were optimized for interactive, incremental workflows — and notably, Cursor and Claude Code do not call themselves “AI-powered software engineers.” Anyone who’s worked in software knows there’s far more to the job than code generation. Coding agents have made engineers who adopt them dramatically more productive, but that productivity comes from the collaboration pattern, not from replacing the engineer.

The same pattern shows up everywhere

This isn’t unique to coding. Consider slide generation — one of the most frustrating agent experiences out there. We hate making slides and would love it if an agent could take it off of our hands. Unfortunately, it just doesn’t work because the dominant approach is to hand an agent a prompt and get back a finished deck. The result is almost always unusable. The layouts are generic, the content is vaguely relevant but never quite right (and usually a mangled version of what you put in), and by the time you’ve fixed everything that’s wrong with the output, you might as well have started from scratch.

Contrast that with working Claude’s integration inside the Powerpoint app. The collaboration mode is that the agent makes incremental improvements based on your feedback. Change this layout. Tighten this bullet. Make this chart clearer. That workflow actually produces something useful, because the human is driving the decisions that require taste — what story the deck tells, what level of detail matters, what the audience cares about — and the agent is handling the mechanical execution that’s tedious and slow for a human. That balance of responsibilities is critical to creating useful outcomes.

This principle is incredibly important. It’s one thing if you’re doing a trivial task like summarizing a Slack thread and creating a ticket for the engineering team to execute on – you don’t need human taste and judgment most of the time. But when it comes to cold outbound into a selective customer pool, making customer-facing slide decks, or programming, the value of human taste has not diminished in the slightest. If anything, when the cost of generating most work is zero, results that have good taste stand out more than ever.

This is what separates agents that people actually rely on from the ones that get tried once and forgotten. The difference has a name — or at least, we think it should.

What it means to be agent-native

Being agent-native means building agents that effectively automate the tedious parts of humans’ jobs while letting human creativity and judgment shine. It means starting from a precise understanding of what your agent is uniquely good at and what your users are uniquely good at, and designing the interaction to take advantage of both. It means being disciplined about what your product should do and what it should absolutely not be doing.

That sounds simple, but it’s actually the opposite of how most agents get built. The default instinct is to automate as much as possible — to position the agent as a replacement for the human workflow rather than a tool that reshapes the workflow around each party’s strengths. The result is agents that try too hard to be like humans: they take on too much scope, they lack the context that makes human judgment valuable, and they give users too few opportunities to steer.

Taking a step back, this instinct is understandable. For as long as we’ve been building software, the goal has been to do what humans do, but faster. Email sequencing tools automated tedious button-pressing. Tab complete helped engineers type what they were going to type anyway. Those tools worked because the task was narrow and the right answer was obvious — there was no judgment involved, just speed. The mistake is assuming that the same philosophy scales to agents operating over tasks where context and taste actually matter. When you ask an agent to write a cold email to a prospect or generate a slide deck for a board meeting, you’re not asking it to do the same thing faster. You’re asking it to make decisions that require context it doesn’t have.

The best agent builders we’ve talked to share a different philosophy. They’ve understood that modern LLMs have an incredibly deep but somewhat narrow intelligence, and they’ve thought carefully about where the handoff between agent and human should happen. They’ve designed their products so that when things go wrong — and things always go wrong — the human can course-correct quickly rather than discovering the problem after the agent has gone down a dead-end path. They’ve built trust through narrow, reliable capabilities before expanding scope.

When you build something that’s agent-native, you have the opportunity to genuinely delight your customers — not because the agent does everything, but because it does the right things exceptionally well and makes its users better at everything else.

The Agent Native series

We’re spending time talking with founders and product leaders at agent-native companies across a range of product verticals. The Agent Native series will share what we’ve learned from those conversations — how the best builders think about the division of labor between agents and humans, how they design for trust, and what they’ve learned about where agents should and shouldn’t operate.

Over the next few weeks, we’ll be publishing posts in collaboration with each of these teams. We’ll share a little bit of background about the problem space, their underlying product philosophy, and what lessons we’ve taken away for our own day to day work. Stay tuned!

The core challenge we face is how we can efficiently validate the work that an agent has done. This problem is lost amongst all the discourse around agents curing cancer and taking jobs. Independent of any future advances in model and agent quality, it is already true today that coding agents, document generators, and AI co-workers have made work cheap. What they haven’t done is make it any easier to know whether the thing that was created is actually right. We’re not talking about code quality or best practices — we’re talking about the basic question of whether the output does what you intended. And right now, the answer is that most people don’t have a good way to check.

The bottleneck has moved. It used to be creation; now it’s validation. And if you’re not careful, you’re just replacing one form of slow, expensive work with another.

Why validation is so hard

The obvious explanation is that validation takes time — someone has to review the output, and at the volume agents produce, there simply aren’t enough hours in the day to do that. The rule of thumb used to be that one person might manage 5-7 employees – now you can manage as many as you can keep track of. Except those employees start from a blank slate with every task and only get better when new model updates are released.

That’s painful enough, but it’s perhaps not the deepest problem with validation. The more insidious challenge is that most people can’t validate agent output even in principle, because they don’t have enough clarity about what they wanted in the first place.

Think about how UI work used to happen before LLMs. A designer would spend days thinking through a feature — mapping out interactions, edge cases, and state transitions. They’d produce detailed mocks and hand them to an engineer. When the engineer delivered an implementation, validation was straightforward: does this match the mocks? The mocks were a checklist. If the designer had missed something, the engineer would probably realize that during implementation. The designer had done much of the hard thinking about UX upfront, and that clarity made it possible to evaluate the result quickly and confidently.

Now imagine you’re building the same feature, but instead of going through that process, someone types a loose description with missing details into a coding agent and gets a working UI back in minutes. There are no mocks to check against. The agent made dozens of decisions about interaction patterns, edge cases, and visual hierarchy that nobody specified. The output looks complete — it runs, it renders, you can click through it — but the completeness is an illusion. The agent didn’t resolve any ambiguity thoroughly; it just papered over it with plausible defaults. And now you’re stuck trying to figure out whether those defaults were good ones, without any reference point for what “good” was supposed to look like.

Before agents, ambiguity in your thinking got resolved during the work. When you mocked features or wrote code, you encountered edge cases and made decisions about them as you went. When you built a financial model, you were forced to confront your own assumptions cell by cell. The work itself was a forcing function for clarity. Agents have removed that forcing function, and nothing has replaced it yet.

This is why clarity of intent has become so critical — both because it makes agents produce better output and because without it, you can’t evaluate the quality of what you get back. If you knew exactly what you wanted, validation becomes something like a checklist: did the agent match these assumptions, handle these edge cases, produce these outputs? You can move fast. But if you went in with a vague sense of direction and let the agent make a hundred small decisions on your behalf, you’re now stuck trying to reverse-engineer whether each of those decisions was a good one. If you don’t start with clear thinking, it can often be harder than doing the work yourself.

The analogy we keep coming back to is delegating to an extremely intelligent college grad with opaque judgment. Whether an agent has good or bad judgment is subjective, but what matters is ensuring that the agent understands exactly what it’s supposed to do – and executes on it. When it’s only humans working on something, the back and forth between a group of people helps discover and test assumptions in the specification. Now, the model is going that all internally – the end result might be great, but not great in the way that you need.

The tools aren’t ready

Even for people who do have clarity of intent, the tools and workflows we rely on aren’t designed for agent-speed validation. Think about the financial model example again. You know what assumptions the model should encode. You know what the outputs should look like for a few test cases. But spreadsheets don’t give you a way to express those expectations and check them programmatically – or frankly even quickly for that matter. You’re stuck eyeballing formulas or plugging in test numbers manually, essentially doing the same work you always did.

The same pattern shows up in software engineering, where it’s even more acute. Previously, you thought about edge cases and validated your assumptions while writing code. That consideration happened incrementally, spread across hours or days of work. Now an agent collapses all of that into a single output, which means all of the validation that used to be distributed across the creation process gets compressed into the review phase. The result is that code review — which used to be a manageable check on largely human-produced work — has become the load-bearing wall of software quality. There’s no chance it lasts.

There’s no incremental testing when an agent does a task, because it delivers everything at once, almost instantaneously. If you skip the validation step or do it halfheartedly, you’re consigning yourself to shoddy results that you won’t discover until something breaks downstream. And if you do the validation step thoroughly, you’ve burned most of the time you saved by using the agent in the first place.

The interaction mode has to change

We’ve established that validation is more necessary than ever and also that it’s harder than ever. We’ve also established that existing modes of validating work aren’t going to work as we dramatically scale how much there is to check. This creates a question we haven’t seen anyone answer well yet: How should validation actually work?

Consider the financial model example again. You know what assumptions the model should encode, and you know what the outputs should look like for a handful of test cases. But the way you interact with a spreadsheet today is by opening cells and mucking around with value — a mode designed for a world where a person carefully built the model and you’re spot-checking their work. That mode collapses when an agent built the entire thing at once and every cell is a potential surprise.

What you actually want to do is interrogate the model. Ask it questions – what happens to revenue if churn doubles? What’s driving the margin assumption in Q3? If the answers match your expectations, you build confidence. If they don’t, you’ve found the problem without having to reverse-engineer deeply linked formulas. The validation step becomes a conversation with the output, not an inspection of its internals.

Not every validation step is going to look like a conversation, but similar principles apply to code, documents, or designs. When an agent generates a complete feature, you need to see whether the UX that was implemented matches what you had in mind. When an agent writes a report, you need to understand what the thesis of the document was and how that argument was validated. The UX of agent-assisted work needs to be built around this kind of fast, targeted verification, not around the line-by-line review workflows we inherited from a world where humans did the creating.

Today, almost none of our tools support this. Spreadsheets don’t let you express expectations and test them. Code review tools are built for diffing human-authored changes, not for interrogating agent-generated systems (where PRs are quickly ballooning in size). Document editors assume you’ll read and redline, not ask and verify. The interaction mode hasn’t caught up to the speed of creation, and until it does, validation is going to remain the thing that eats all the time agents save.

What this means

We think this is among the most important unsolved problems in AI tooling right now – and it’s barely even being discussed. The industry has invested enormously in making generation faster and cheaper, and it’s worked. But generation without validation is just moving the bottleneck, not eliminating it. People are going to be overwhelmed by agent output, and they’re either going to be disappointed by the quality of what ships or they’re going to stop checking and be surprised when things break.

The UX of AI-assisted work has to change to match this reality. It can’t just be about faster creation; it has to be about faster verification. That might mean agents that validate their own output against explicit expectations. It might mean production tooling that watches for the consequences of bad agent work in real time. It will almost certainly mean both, and probably things we haven’t imagined yet.

But one thing is clear: Without solving validation, we haven’t actually solved the productivity problem. We’ve just moved it.

This distinction sounds simple, but it has profound implications for how you build agents — and how you talk about them. The right way to design an agent is to start from what humans struggle to do at scale: process large volumes of data without fatigue, consider many possibilities in parallel, or find an interesting detail to personalize sales outreach.

This is a critical distinction in how you explain, build, and iterate on your product. You can choose to focus on cooperating with humans – an agent designed around what humans can’t do and built to hand off gracefully when things get messier than expected. Or you can choose to copy-paste a human job description into a product spec with a rigid set of requirements, something that will inevitably disappoint customers with mismatched expectations.

The AI industry has, almost universally, chosen the second path. You can’t take a step in San Francisco without seeing a billboard for an AI SDR, an AI SOC analyst, or an AI SRE. We understand why — customers are searching for these terms, and if your website doesn’t speak their language, you’ll lose the SEO battle before you get to make your pitch. But we think the race to anthropomorphize agents is setting the entire industry up to fail.

Why job-title naming creates a trust crisis

When you name your agent after a job title, you’re making an implicit promise: that your product can do what that person does. That includes everything that never appears in the formal job description.

Take an AI SDR. The pitch is intuitive: identify the right prospects, send them the right message, and generate pipeline. In our experience, agents are genuinely good at account research, ICP matching, and identifying the right person to contact at a given company. Where they fall short is the second part. What a customer buying an AI SDR expects is to turn it on, feed it some messaging, and come back to a healthy pipeline. What they mostly get is good targeting attached to generic outreach that doesn’t convert.

The gap is the messy bits that humans often intuit well. Good outbound sales requires creativity and the ability to build rapport with a stranger in a single sentence. Humans who are good at sales have spent years developing that instinct. Agents, today, produce messaging that sounds automated, and we’re all inundated with that nonsense. It’s easy to imagine an agent that does all the research and hands off to a human to get the prospect over the line. But that’s not the expectation the name set.

We could make this argument across every anthropomorphized agent market. In our own experience, the phrase “AI SRE” is frankly a poor description of what our product actually does — but it’s what customers are searching for, which creates a genuine tension we’ll come back to.

What good naming looks like

The most instructive counterexample is coding agents. Cursor didn’t call itself an AI software engineer. It started off as an AI-powered IDE — a tool that generates code effectively across a complex codebase. They didn’t set the expectation of being an autonomous engineer that ships features end to end.

This wasn’t just semantic humility. The people building these products had a good understanding of software engineering to know that everything a software engineer did was not immediately in scope for automation. Humans would still be responsible for the architectural decisions, the cross-functional coordination, the customer conversations that shape what gets built.

By scoping the promise to a task – generating code – rather than the role, they gave themselves the opportunity to match (and beat!) users’ expectations. Engineers who adopted these tools didn’t feel like they were being replaced. They felt like they were getting faster at the part of their job they liked least. Over time, trust built through thousands of small interactions where the tool did what it said it would do. That trust is what eventually opened the door to agents taking on more.

It’s not a coincidence that coding agents have the deepest adoption, the strongest data flywheels, and the most widespread acknowledgment of quality in the entire AI application landscape.

What agents should actually be built for

The deeper issue with anthropomorphizing isn’t just the naming — it’s that job-title thinking constrains product design in ways that limit what agents can actually become.

Most agent products today are built around a simple idea: observe what a human does, then automate it. For an AI SRE, most products are asking their customers to write a runbook for every alert type. Then they build an agent that follows the runbook, posts a summary in Slack, and pretends to have capabilities that auto-remediate an issue. This approach produces passable results but creates tons of process overhead that users quickly learn to ignore – and it doesn’t create the opportunities for users to actually trust the end-to-end resolution workflow. This is marginally better than the human workflow it replaces, but you’ve left most of the agent’s actual capabilities on the table.

The more productive question to ask is what can an agent do that a human fundamentally can’t? Not something that an agent is slightly faster or cheaper at — what is truly different about an agent’s capabilities? Continuing the AI SRE analogy, an agent can monitor thousands of signals simultaneously without fatigue. It can run dozens of hypotheses in parallel and discard the ones that don’t pan out. It can surface patterns across an entire customer base that no individual human would ever accumulate enough exposure to recognize.

This realization meaningfully changed how we’re building RunLLM. Rather than following fixed alert thresholds the way a human on-call engineer would, we detect incidents by analyzing the frequency and content of observability data to surface anomalies before they’ve been formally flagged. Rather than working through a runbook linearly, we spin up multiple subagents in parallel to evaluate many hypotheses simultaneously. And rather than claiming that we’re going to automate end-to-end resolution, we give users the option to pick the handoff point that they prefer. The agent does not try to map a human workflow; it focuses on automating things that humans can’t do and letting humans handle the rest. That’s why our product delivers outcomes that a copy-pasted human workflow never could.

Where this is all going

The implication of this argument extends beyond product design and naming. If agents are genuinely shallow and wide — good at the learnable, scalable slice of many adjacent roles — then the natural endpoint isn’t one agent per job function. The boundaries between traditional roles will naturally become less fixed over time. Humans already manage multiple agents across overlapping domains, and that will only increase.

A software engineer who uses an SRE agent to monitor production isn’t doing an SRE’s job. They’re doing their own job better, with more visibility. An SRE who uses a coding agent to patch a bug during an incident isn’t moving into software engineer. They’re resolving the incident faster. The agent isn’t replacing either person — it’s dissolving a boundary that only existed because humans have limited bandwidth.

The companies that stop asking “which human job does this agent replace?” and start asking “what can this agent do that the human couldn’t?” are going to build products that are genuinely differentiated, not just incrementally better. The rest will keep fighting over SEO terms while quietly underdelivering on the promises those terms imply.

We were fascinated to hear about their experience. Despite being an engineer at one of the most technically sophisticated companies in the world – one that builds its own foundation models! – they encountered significant and widespread reluctance to adopt coding agents. Some engineers simply liked writing code by hand — it was part of how they thought about their craft. Others couldn’t figure out how the tool was supposed to fit into their workflow. Still others had tried it, found the UX confusing, and quietly moved on without sharing any feedback.

This isn’t a story about backwards-thinking people who aren’t ready for change. These are excellent engineers at a company that’s been at the center of technology for decades – if you saw this name on a resume, you would immediately consider interviewing the person. And yet, for most of them, AI coding tools remain something they’ve heard about but haven’t really integrated into how they work.

If that’s true there, imagine how much of a challenge this is everywhere else.

The bubble is smaller than you think

If you’re reading this blog, you’re almost certainly deep in the AI bubble. You’ve probably used Claude Code or Cursor to build something from scratch on a weekend and told everyone you know about it. You’ve definitely seen the LinkedIn posts — the ones breathlessly explaining that a repo with Claude Code skills for go-to-market is going to revolutionize the way you do sales. We get it. We write this blog, after all.

The problem is that spending time inside the bubble makes it easy to mistake the bubble for the world. The users who are most excited about AI — the ones tolerating rough edges, experimenting constantly, building their own workflows — are a small fraction of the people who are eventually going to need to use your product. And crucially, their enthusiasm can paper over a lot of gaps. If someone is genuinely excited to use your product, they’ll push through a confusing onboarding flow, forgive a clunky UX, and figure out the right prompting pattern through trial and error. You can build a real business on these true believers for the first little while – and they’re genuinely wonderful people to work with.

But you will eventually have to account for the rest. And we think most AI companies aren’t building for them.

Why the gap is a product problem, not a people problem

The instinct when faced with reluctant users is to assume they’ll come around — that adoption is just a matter of time, and the product doesn’t need to change if its working well for the true believers. AI adoption has a different quality from previous technology changes, and the reason is rooted in something more fundamental than UX.

User identity

AI tools attack user identity in a way that most software doesn’t. Think about what it means to be a software engineer who has spent years taking pride in the quality of the code they write. Now an agent is generating code at 10x the speed, and most of it looks like slop to them. For many senior engineers, it’s probably true – they could write better code if they took their time. More importantly, that code is an affront to how they see themselves professionally. They’re never going to merge this code without a close inspection, no matter how many benchmark improvement graphs you put in front of them.

But here’s the flip side, and it’s where the real product opportunity lies: The best AI products have the opportunity to actively reinforce user identity too. Done right, your product doesn’t make the careful engineer feel replaced; it makes them the most forward-thinking engineer on their team. It doesn’t make the skeptical PM feel like they’re cutting corners; it makes them the person who shipped a working prototype before anyone else had finished writing the spec. The goal isn’t to neutralize the identity question. It’s to make your user the superstar of their organization, in a way that feels consistent with who they already are.

Getting there requires meeting users where they are and bringing them along gradually. The coding agent world offers the clearest model of how this works in practice. The trust that made widespread agent adoption possible wasn’t built by agents — it was built by tab completion. Engineers who would never have handed off a function to an autonomous agent were perfectly comfortable accepting a 1-5 line suggestion. That comfort, built up over thousands of small interactions, is what created the foundation for people to trust agents with more. It’s not a coincidence that the products with the deepest agent adoption today are the ones that also had the best tab completion two years ago.

Editorial judgment

The corollary to meeting users where they are is not overwhelming them with everything at once. This has always been true in product, but AI has made it a more acute problem. The ability to ship new features quickly — which AI has genuinely accelerated, and AI products are taking full advantage of — makes it easier than ever to pile on functionality before users have developed trust in the core experience.

A reluctant adopter who opens your product and finds a sprawling set of capabilities they don’t understand isn’t going to dig in and figure it out. They’re going to confirm their suspicion that this thing isn’t for them, and they’re not coming back for a long while. The same logic applies at a feature level: if a user is still getting comfortable with tab completion, immediately introducing an autonomous agent mode isn’t going to accelerate their adoption — it’s going to spook them. Doing less, more reliably, for the right person at the right time is how you earn the trust that eventually lets you do more.

This concern applies in many other domains. We see every day that most engineering leaders are only now coming around to the idea of having an AI SRE help them manage production software – and while these folks are figuring out what they want, vendors are promising that they’ll autonomously make changes to your production infrastructure while you’re still asleep. That might sound exciting to the most ardent believer, but it sounds like absolute insanity to most people. Most products in the space haven’t figured out the balance yet.

The maturation moment

There’s a useful historical parallel here. In the early days of cloud computing, the customers were startups and tech-forward companies who were willing to figure things out as they went. AWS in 2008 was powerful but rough, and the people using it had enough technical sophistication and tolerance for ambiguity to make it work. Then came the maturation phase: large enterprises, regulated industries, banks, healthcare systems. These customers had different expectations, different risk tolerances, and different definitions of “good enough.” The cloud providers that thrived were the ones that understood the product had to change — not just get cheaper or faster, but fundamentally adapt to a different kind of user.

AI is entering the same phase. The early adopters — the ones giving talks to thousands of engineers, the ones building apps on weekends and posting about it — have gotten the industry here. The next phase of growth is the middle 50%: the users who are curious but skeptical, willing to try but quick to bounce, not interested in figuring out the right prompting strategy on their own. That is the audience that’s going to determine whether AI actually delivers on the productivity promises that have been made on its behalf.

What makes this moment unusual is that unlike the cloud, AI seems to be compressing the early adoption and maturation phases together. The gap between this is a tool for enthusiasts and this needs to work for everyone is closing faster than anyone expected. For product builders, that’s both an opportunity and a warning. The companies that figure out how to cross the chasm early are going to have a significant head start on the ones that are still optimizing for the bubble when the rest of the world shows up.

While the promise is appealing, the reality is not. We think the companies building in this space are walking into a trap — and it’s worth understanding why.

The promise of horizontal intelligence

Imagine a finance team trying to assess the risk profile of outstanding invoices. Correctly determining what the aggregate risk of these unpaid invoices means pulling together bank data, invoicing tools, customer contracts, historical payment records, and the latest usage data. Doing that by hand is tedious, error-prone, and slow — exactly the kind of workflow that a well-built agent should be able to automate. And any given company has tens or hundreds of these tasks a day, each one of which is bespoke and repeats only rarely. It’s obvious why you’d want to build a startup around this problem.

The horizontal intelligence pitch is essentially this: We’ll connect to all your data, understand who you are and what you have access to, and surface the right information at the right time. That’s a genuinely useful tool, and we’re not arguing otherwise.

What we’re arguing is that it’s an extremely difficult place to build a defensible startup.

Why horizontal intelligence is a trap

The risk that’s becoming increasingly hard to ignore is that the foundation model providers are coming for this space themselves — and they have a built in head start.

Each of the major providers has a slightly different wedge. Google owns Workspace, and while the Gemini integration isn’t yet great, odds are that it will improve significantly. OpenAI, through their apps framework, is increasingly able to integrate across a diverse set of tools and has built a developer framework that encourages application providers to do the heavy lifting of integration themselves. But it’s arguably Anthropic that’s furthest along: the Claude desktop app is genuinely impressive, and the integration of Claude Code and Cowork into the local file system — editing documents, analyzing data, generating artifacts — clearly indicates where this is going.

The uncomfortable question for horizontal intelligence startups is why will your product handle enterprise data better than the major model providers? We have yet to hear a compelling answer to that.

There are two interesting things about this kind of task that give the model providers a clear advantage. First, there’s relatively little specialization to this kind of intelligence. Every task is unique, and it’s often the case that once you have the right data, it’s relatively easy to solve the problem – the data is just all spread all over the place. Second, pre-built templates like slide formats and document structures are starting to become standard agent inputs. Once these are agreed on, all that’s left is data connectors, search, and general intelligence – all things that the model providers have in spades.

The products that hold the actual underlying data are going to feel increasing pressure to make it available to the model providers’ agents — or risk being displaced by something that is more LLM-friendly. That’s not a dynamic that favors a startup sitting in the middle.

Why specialized intelligence wins

None of this means there’s no room for startups in the agent space. Quite the opposite — we think there’s a significant opportunity. It’s just not in the horizontal middle ground between model providers and data stores. The opportunity exists in workflows that require specific expertise and customization that are genuinely difficult to recreate without encoding that intelligence into the architecture of a purpose-built agent.

The reason specialized vertical agents can win where horizontal agents can’t boils down to a few related ideas.

First, when an agent makes a strong set of assumptions about the workflow it’s solving, it becomes much easier to encode intelligence into how the agent actually operates. There’s a meaningful difference between “summarize this document” and “execute this specific workflow for this engineering team in this company.” The former is something a general-purpose model will get better at over time. The latter requires a level of domain-specificity that can’t be learned from generic usage data – a combination of company-specific context, general engineering principles, and the ability to learn and improve over time.

Second, that narrower focus allows you to solve increasingly hard problems. We’ve all had the experience of asking a general-purpose agent to do something complex, only to find that it didn’t quite understand what we wanted and produced something generic and likely useless. A specialized agent can enforce much stronger guardrails on what gets generated — it knows what good looks like, and it can be built to enforce that rather than relying on general-purpose reasoning and hoping for the best.

Third, and most importantly, this specialization creates a data flywheel that the model providers can’t easily replicate. The model providers have enormous amounts of generic usage data. But in order to bring that data to bear on complex, domain-specific workflows, they also need to know what a good outcome looks like — and for enterprise-specific workflows, that ground truth simply isn’t available to them at scale. The more complex the task, the harder it is for a foundation model to build useful generic intelligence around it. That’s where the moat lives.

We’ve written before that data is the only moat available to most AI startups. Vertical agents are the clearest expression of that thesis. The data flywheel a vertical agent builds takes two forms: a large volume of broadly applicable data about one particular kind of workflow (which generalizes across customers), and deep data about how one specific company does that workflow (which makes the agent increasingly irreplaceable within that account). Both are difficult for a generalist to compete with.

Picking a lane

Our conviction, built from a few years of building in this space, is that the companies that are going to win are the ones that pick a lane and go deep. The less an LLM can solve your problem off the shelf, the better your defensibility over time.

That’s exactly why we’ve focused on what we’ve called hard-hard problems — the enterprise workflows that are technically complex and organizationally difficult to adopt. These markets are daunting. In many of them, startups have raised significant capital without anyone running away with the space yet. That’s not a sign that the problems can’t be solved. It’s a sign that the model providers can’t solve them generically, and that the engineering required to do it well is genuinely hard. That difficulty is exactly the moat you want.

If you’re building an agent today and you find yourself describing your product as an intelligence layer, or as a way to connect enterprise data to a model, we’d encourage you to stress test that framing. The foundation model providers are building the same thing — and they have more data, more compute, and more distribution than you do. The only place that argument doesn’t apply is in the workflows that are too complex, too specialized, and too company-specific for them to tackle. That’s where we are building.

The bear case for software-as-a-service goes something like this: As coding agents improve, the build-versus-buy calculus shifts permanently. An engineer at any company can now pick up a coding agent, build a prototype that meets the company’s specific needs, iterate on feedback, and deploy a bespoke tool that provides more value than a generic SaaS platform at a fraction of the cost.

The source of this concern is our collective recent experience with the improving quality of coding agents. We have all had Claude Code or Cursor scaffold an entire application from scratch for a few dollars in tokens. When you see something genuinely useful built in minutes, it is easy to assume the multi-billion-dollar incumbents are doomed. And if an individual can build a prototype quickly, surely a startup can build a strong offering with a few months and a few engineers?

As with most hype cycles, however, the truth lands in the middle. While there are massive opportunities for disruption, the SaaS business model is not going to evaporate overnight. In our view, the distinction comes down to a quick survival checklist. If you meet one of these criteria, you are likely safe. If you miss on all of them, you should be worried:

1. Are you a system of record?

2. Do you do more than help humans automate a single workflow?

3. Are you mission-critical?

The Physics of Data Gravity

Data moats have long been the holy grail of enterprise software. Consumer products like Google or Instagram win on user behavior data, and enterprise titans like Snowflake or Datadog are powerful because they are systems of record. These companies do not just power workflows; they house years of historical data that companies have imported and structured.

Despite all the massive technological shifts in the last few years, the physics of data have not changed. Moving massive amounts of data is expensive, risky, and slow. This means that, as has been the case for fifteen years, the major cloud providers will continue to make their margins on networking and egress. The cost of moving data in and out of a third-party service — or your own cloud — remains incredibly high.

Beyond the operational cost, there is the issue of operational stickiness. If a product is collecting data for a core operational purpose, it becomes a load-bearing wall in the company’s architecture. You do not pay Datadog or Snowflake eight figures a year because storing logs is a nice-to-have feature. You pay them because when a mission-critical issue happens at 3:00 AM, you need a proven way to investigate, pinpoint, and fix the issue.

But realistically, your business will continue to exist if Datadog goes down for a little while. Things get much more difficult when we’re talking about truly mission-critical software.

The Mission-Critical Tautology

Even when companies don’t have the most interesting data, they’re likely safe if they are critical to the operations of their customers – the kinds of products that your business literally wouldn’t exist without. The obvious examples are Workday and Salesforce.

There is a certain recursive logic here: These companies are safe because they are big, and they are big because they are safe. No matter how shiny a rapidly scaffolded payroll system looks, a VP of HR is not going to risk payroll failing on a Friday morning. A CRO does not care how many new automations a startup offers if it means moving away from a Salesforce instance they have spent a decade customizing to their exact sales and revops motions.

When enterprises make these decisions, they are not just buying software – they are also offloading risk. A tech-forward company like Google or Meta certainly has the technical talent to build an internal payroll system. The question is not whether they can build it, but whether they want to own the risk of running it. Once they do the calculus, dedicating hundreds of engineers to a non-core area of expertise rarely makes sense.

By contrast, non-mission-critical software is in the danger zone. Analytics is the prime example. Traditionally, writing code for dashboards and plumbing database queries was prohibitively expensive to democratize.

Just last week at RunLLM, we built an internal dashboarding system following a conversation about product metrics. We used Python and Plotly to connect to internal systems and pull customer data via our CRM’s API. If this dashboard goes down for a day, it is annoying, but it does not halt our operations. More importantly, because the cost of building it with a coding agent was so low, and the upside of allowing our VP of Sales to modify it himself was so high, the option to buy a traditional BI tool never even entered the conversation.

The Workflow Moat Narrows

Pure workflow software is where we’re currently the most bearish. By workflow software, we mean anything that’s focused on connecting the dots between existing tools that either have data gravity or mission-criticality. The poster child on X for this is PagerDuty. At its core, PagerDuty processes data stored in another product’s telemetry store, determines if a threshold was met, and alerts an on-call engineer. This is connecting the dots between Datadog and Slack. While PagerDuty does much more in practice, that primary workflow is what most customers buy.

These are exactly the kinds of integration code workflows being replaced by agents today. The threat here is not just that agents will automate incident management or sales outbound; it is that new players or internal teams can build superior user experiences that blend human and agent capabilities from the ground up. The most interesting part of this comes from the fact that the solutions can (and should) very easily be customized to match each team’s workflow. These agent-first systems will be dramatically more valuable because they save human hours, easily displacing legacy systems that are human-first and currently scrambling to bolt on AI features as an afterthought.

The Ops Gap

The undiscussed concern underlying all build it with Claude Code software is operations. The reality of software operations and reliability is unfortunately still incredibly complex and oftentimes dwarfs the time taken to build something useful. The gap between building and operating is actually growing because as coding becomes a commodity, the relative cost of infrastructure and security grows.

The internal dashboard we built recently is a perfect example of this. It took about an hour to iterate on the core dashboards themselves with Claude Code. It then took about four hours to deploy it in Google Cloud so that it was properly authenticated behind our company’s SSO, set up to auto-update with new commits, and connected with the right credentials.

This gap will likely persist. The simple reason why is that Python looks the same no matter where you work, but infrastructure is dramatically different at every company. There are not yet great ways to deploy cloud software without getting into the weeds of Kubernetes clusters, networking permissions, and security + compliance. None of those things matter when you are prototyping locally, but they are the only things that matter when you are operating production software at scale.

Wrapping Up

The rumors of the demise of modern software are greatly exaggerated, but the nature of what makes software valuable is quickly changing. For the last decade, SaaS companies could survive by being a slightly better UI for a human workflow. That’s no longer defensible.

The defensibility remains exactly where it has always been: at the intersection of data and risk. If you own the system of record, the physics of networking and the high cost of data egress will protect you. If you own a mission-critical operational process, the corporate aversion to risk will protect you.

The real danger is for the middle layer of the stack — the products that have built businesses around the friction of human work. As the cost of generating code and automating workflows drops toward zero, the value of that software must move elsewhere.

This does not mean incumbents are invincible. It just means that the disruption will not come from a simple internal tool or a weekend project built with a coding agent. To win in this new environment, startups must figure out how to best manage the data, reduce the operational risk, and close the gap between a prototype and production-grade reliability.