Before agents, I got my reps as part of writing code: try different approaches out, debug what went wrong, review other’s code, read a lot. Agents can skip much of that work, so building your reps has to be deliberate.

If I was new to the industry, I’d try to form a hypothesis before prompting. Ask “why” a lot, read the diffs, try to predict what might fail. Occasionally try to work through the problem myself manually.

In my experience, good agent work depends on two abilities:

1. Deep expertise: you understand the problem domain well enough to define a good outcome. Understanding your user/product/business is part of this.

2. Applied judgment: use your taste to turn this into a clear, testable plan by choosing the right context, constraints, tests and verification.

To build these the skills I’d practice are decision making, specifying, steering and verifying.

The reps I actually did

When I got started in software engineering, I very much felt like I had no idea what I was doing. I was having a lot of fun building, a lot of fun trying things out, failing, learning from my mistakes, and each time getting a little further and further in my journey. And every time I failed, I tried to take that as another building block in becoming a better engineer. And so all of this work was me doing the reps. It’s the way that I learned JavaScript. It’s the way that I learned how to program in C++ and build desktop applications. It’s how I learned how to tune the performance of graphics intensive applications, all of these types of things.

Building up the reps, very often I would go into a task with some sort of hypothesis or an idea, even if it was, like, super wrong about how things might work. I would try out what I thought could work, and when it didn’t work, I would then go to Stack Overflow or search the web for different documentation and absorb some knowledge. Maybe read some books if it was a very esoteric topic. I would then continue on my journey. And you do that enough times and you start to build out expertise, especially once you start doing this for real and trying it out on real world projects that go beyond hobbyist stuff that you might be doing at the weekends.

Most of the judgment I use today came from thousands of small reps like these: debugging failures, reviewing other people’s code, and living with abstractions that looked good until a real system pushed back. Agents can now skip much of that work. If you’re three years into your career, plausible code may arrive faster than your ability to judge it.

The short-circuit

Now, I think that for many people who are getting started with AI, you can short-circuit a lot of the learning journey. You can go very quickly from, hey, here’s a problem, to, well, hey, here’s the solution, or here’s the outcome of the overall task, while skipping all of those things that would otherwise have built up your knowledge base, or helped educate you about, you know, don’t do that thing, this is why you don’t do that thing, do this thing, and help you reason about the trade-offs. And I think that this is one of those areas where it’s going to require junior engineers especially to be proactive about their educational journey.

I’ve also talked to a few different AI labs. Many of the main players in AI right now are very focused on helping you accomplish an outcome or get an answer as quickly as possible, and they don’t necessarily help you in your education journey unless you’re specific about that being one of your goals. Like, there’s a difference in me saying, hey, help me build an app for scheduling, and, help me build an app for scheduling and teach me how to do it as we’re going, going one step to another. Most people don’t do the second one of those things. And part of it is not knowing that that is an option. Part of it is perhaps thinking, well, hey, these days there are all these velocity expectations and there’s this pressure to ship fast and just move on to the next thing. But I do think that in order to become better engineers, to continue having this expertise that improves our taste and our judgment, you do have to go out of your way to build up mastery.

A completed task is not necessarily a rep

I think that with any kind of critical thinking, with any type of problem that you have, it’s useful to have a hypothesis about the solution, an idea about what it might look like: the shape of it if we’re just talking about logic and code, how it might look and feel and interact if it’s a piece of UI. When a task is finished, it doesn’t necessarily mean that you have learned something. It just means that the task has been finished. You have to almost look out for those learning opportunities, or you can ask your agent to summarize as it’s building or at the very end: what are the key learnings from this that would help me as an intermediate developer, or as a junior developer, increase my knowledge base or improve how I think about problems? And you can keep doing that. You just have to be proactive about your learning journey.

For me, there are several things that I’ve been able to use AI for these days, and more complex 3D graphics programming is definitely one of those. I’m not an expert. And there are definitely times when I try to make sure I’m asking the AI, okay, so can you explain how this thing works? Can you teach me about this concept you just implemented? Can you help me reason about how these different elements connect? And I think that because I want to learn, and I have that desire to learn, I am pairing with my agent in order to do that. If you’re not necessarily trying to learn, you lose opportunities there.

A completed task not being a rep is also something that happens when there aren’t mistakes in the process. When there are mistakes, you start to think, okay, well, why did it go wrong? What could be better? What am I not thinking about? And it forces you to reflect. When things go right, there’s not really a teaching moment there. You just think, okay, well, the work’s done, I’m just going to move on to the next task. And so, especially if you’re junior, you want to be looking for those opportunities to keep leveling up.

There was a 2026 study by Anthropic looking at junior engineers learning a particular Python library, Trio. People who used AI assistants scored 50% on a follow-up quiz against 67% for the group who were working by hand. And within the AI group, the strong results came from those who asked conceptual questions and requested explanations rather than treating the model as a code vending machine. This ties back to what I was saying: if you are just using AI to generate output and generate outcomes, but you’re not using it as a pair, you’re not using it to try improving your critical thinking skills, your knowledge skills, your understanding of how things work, you can end up in this situation where you largely don’t understand how things work, but you’re just good at prompting. And that means you’re perhaps not really going to be so good at the verification side of things. It doesn’t surprise me too much that people who asked questions and requested explanations did better. Those people probably had a lot more reflection on how things worked, how it connects to other things that they know. They pattern match, they start to build up residue about, okay, well, this is how this thing works, this is how I reason about it, these are the gaps in my knowledge. And so seeing that 17% difference kind of makes sense to me. Of course, this was a short-term study of just one Python library, so I wouldn’t say it’s conclusive necessarily, but it was still very interesting.

This is still how I work when I’m learning something unfamiliar. I try to keep myself in the loop. I form a hypothesis before prompting. I ask why, inspect the diff, predict what might fail, and give the agent a concrete way to verify its work. Occasionally I work through a small problem by hand. I want the agent to close the task while my mental model still moves.

Use them aggressively anyway

I don’t know how long code-level expertise will remain as valuable as it is today. Models are improving too quickly for much certainty. I also don’t think the answer is to avoid agents or romanticize typing every line. I use them aggressively. On some days I have five or ten sessions running, and I once caught myself asking the wrong project to add dark mode. That mistake clarified the constraint: agent throughput scales faster than my attention.

A thousand hours in the performance panel

I can think of things that I had to spend thousands of hours to get right. Performance optimization is one of those areas where, back in the day, you didn’t always have a whole lot of great blog posts or books that you could consult. There was some decent, very classic literature on these topics that would maybe touch on memory or how to think about hardware and constraints. But you take something like web performance optimization, JavaScript optimization, heap optimization, all of these things, there weren’t always great articles about these things. And so you would build your reps by going into the Chrome developer tools, using the performance panel to run a trace of a page or an application, interact with it, try to find, like, where is the slowness? And then trying to drill down and come up with a hypothesis of, okay, well, it looks like this is the area of the flame graph where most of the problem seems to be. Or this is where maybe, in the memory panel, I’m not allowing garbage to be collected, or anything like that. In my time, you had to have gone through the gauntlet of making enough mistakes, attempting to find out the root cause, that you built up this knowledge, this esoteric at times knowledge, about what worked and what didn’t.

These days, a similar flow would be one where you’d have the DevTools MCP go and do the performance profiling for you with your agent, and figure things out, and then come up with the fix for you itself. And so you don’t necessarily then build up that expertise in performance quite as much.

You can only prompt what you can imagine

When I scroll through Twitter these days, I am always impressed with how much imagination and creativity is in my feed. So many designers, creative people sharing amazing shaders, amazing games, UI, immersive experiences that they are building that is now even more so possible. Like, the tech was there, but imagination is now the ceiling. It’s much, much more accessible for you to build these things much more quickly. But you have to have that imagination in order to have the idea in the first place and tell your agent to build it. And then you have to have that expertise to verify it. So verification is the floor and imagination is the ceiling.

I remember, for an upcoming album site (I do music), I wanted some of the homepage to be these 3D objects that were interactive, that are part of the experience. Things like 3D CD players, and I think I had a vinyl record player in there as well, maybe a tape player, some 90s nostalgia. Now, the initial versions not only didn’t look amazing, but they didn’t follow the right interaction pattern. They didn’t perform as well on mobile. And so I had to first of all have the expertise to notice that it was buggy in some way. Maybe any user would notice that. But then I had a hypothesis about why that might be. And I could then go and either profile it myself or ask my agent to profile it and figure out what happened, what went wrong. Maybe there was just some way in which the interaction logic was written that wasn’t great. And so I think that your imagination is really important, but then so is your expertise. Both of these things are important. If you can think it, you can make it.

Does the next generation need the expertise?

There is a valid question about, like, hey, if an agent can do these tasks, and increasingly well, do humans need to build up that expertise? Does the next generation need to build up expertise in some of these esoteric areas? And I think that, at least today, where that still becomes useful is places where the agents don’t do a perfect job, where their work does need to be checked. Where you ask something to optimize a particular loop, an animation, a scheduling routine, or anything like that, and maybe it does that at the cost of something else. And if you don’t know what to spot, or you don’t know how to read the implementation and understand what was done, you can end up shipping something that actually doesn’t do what you want.

Skills and MCPs can encode a useful workflow. They cannot tell you when its assumptions no longer fit your system.

There was an Anthropic study of around 400,000 Claude Code sessions that looked at expertise as being this task-specific thing. And it found that having even intermediate expertise about the task that you were trying to complete increased the chances of you reaching verified success with that task, rather than someone who is a little bit more novice. It doesn’t mean that you have to have a decade of experience across the stack, but it does mean that you need to understand the problem domain enough to recognize what good means. We sometimes talk about that these days in terms of taste, and I’ve written about this before. This is also one reason, when I read the Claude Code best practices guide, I’m very happy to see that it starts off talking about verification: testing, using screenshots, other signals that give your agent something that it can continue to iterate against, and gives you evidence to review instead of just some simple summary saying that the task is complete. Having expertise helps you shape clay much better than someone who doesn’t have a lot of expertise but can maybe shape something that looks okay.

And this all comes back to having that expertise to be able to verify the work, to be able to judge the agent’s work. And so I’m hopeful that we can continue to invest in mastery and invest in craftsmanship, even as software engineering continues to rise in the abstractions that we’re using to build software.

The return on expertise is going up

I feel like software engineering fundamentals are going to continue to be important. Expertise is going to continue to be important. And now that the floor has been raised, AI is also increasing the return on the skills that people have, on the expertise that people have. People who are junior stop being junior by shipping real things and making mistakes, learning, building the reps. Experts kind of have an intuition about what to build, how to verify it, how to make sure that you know it’s good, it’s not broken, it’s going to be maintainable, it’s going to scale, it’s going to work in the different contexts or platforms. And especially now that so many people are able to just prompt and bring an idea into being, making it high quality and good enough to ship, delightful, and something that is maintainable and isn’t going to break in production, those skills are going to continue being important.

I run into this at least a couple of times every week. It’s so easy now to prompt any kind of app, any kind of feature. For example, I’m building a text editor at the moment, not from scratch. The idea for this is to be sort of a writing aid that highlights opportunities for your grammar to be better, or to not be using AI style writing, that type of thing. And a frontier model was able to generate me, with a lot of back and forth, a nice and okay looking UI. It wasn’t amazing. And it had a bunch of issues, such as it didn’t have the optimal use of screen real estate. It didn’t have good color contrast. It didn’t have a good scrolling model. All of these things that I know because I’ve made these mistakes before, I’ve built up the expertise. But if you don’t have that expertise, you might just prompt something, put it out into the world, and then stop. And you don’t know what’s better, because you haven’t put in the time to build up that expertise.

Put the lesson where the next agent can find it

One of the things that I tell people I mentor is that when you work with an agent, you should be making it better, and it should be making you better. And what that means is that every day, there should be some sort of cycle where you’re getting things added to lessons or to memory or something so that it’s able to improve. Because otherwise, every time that you’re starting a new session, it can feel like you’re onboarding a new hire that has amnesia. They’re not necessarily going to remember the subtleties of your business, your product, your team, your users, or any of that stuff. And so this is why we end up capturing so much in not just skills, but context and all the stuff that we try to give our agents. And we need to be careful about things that are actually useful and actually specific to problems versus things that we just think are making things better. And so I always encourage people to see, how can you make sure that you are teaching your agent more, and making sure that every day it’s getting better and you’re getting better?

If you are in a chat window and you happen to be solving a problem, like, let’s say that you discover some subtle scrolling bug in a UI component that you’re working on. You work with your agent, you go back and forth, and there’s a lesson somewhere in there that you could potentially use in the future. Now, maybe that lesson will get added to memory. Maybe it won’t. And especially if it’s a long session that has compacting, that full lesson may not necessarily go in there. So that lesson could disappear when the chat window dies. While if you instead try to codify things like specific lessons, tests, lint rules, anything, especially that is small enough that it can be codified in your repo, it can teach future agents. And I found that personally very helpful. If I learn a lesson, I take a few minutes to review and see, is this worth adding to my lessons.md, or asking my agent to add it to its memory, or something that’s just going to keep it sticky? Because I don’t want lessons to disappear. I’m going to forget personally, I’m going to move on to the next problem. This comes up all the time for me. It can be everything from, hey, I have a particular preference for how I approach UI, to how I approach writing components, to how I approach performance, all kinds of things. And if there’s a subtle way in which I address a problem, I want my agent to remember that, or have a way to remember it, rather than me having to continue restating this every single time.

Very often we treat our agents as something that’s going to remember everything that we do, and that’s not necessarily the case. Even if the agent has got a memory system, you can’t necessarily fully rely on it to recall all of the interesting things that you were maybe trying to learn, or the way that you like working, or the way that you would approach verification. And so it is okay to start capturing more of these things in markdown files. Just be very, very careful and cautious that you’re not over investing in that as a strategy. I always liked this idea of a dual loop. A good rep where you learn should sharpen you, and it should sharpen your agent. And when you have some hypothesis that was maybe corrected, you consider if that correction warrants becoming a linting rule, some type constraint, a documentation convention or a test, just so that it can stick around and benefit you in the future.

The outer loop

So I think what all of this means with respect to mastery is: invest in your expertise and in your craftsmanship. Do the reps, make mistakes, learn from them. You will over time be able to figure out what deserves to exist. You’ll be able to start writing up plans, refining plans, coming up with some definition for what done means, and also planning out for those places where humans are going to stay in the loop to check on correctness, safety, or user impact. That is going to be largely the outer loop I think engineers are going to need to own today. We’re going to keep seeing AI moving engineering further up the abstraction layers. And the more agents that I can run, the more care I need to choose where my limited time, taste, and judgment goes.

I feel like there’s been a lot of confusion about skill files and what to do with your CLAUDE.md and AGENTS.md files, especially as I’ve been reading developer discourse on Twitter over the last few months. People have been saying things like, hey, keeping your skills and CLAUDE.md/AGENTS.md files up to date is a big pain point. Very often, people are trying to get them to steer their agents, but are having a hard time keeping them under 200 lines long, even if that’s an official target. People are finding it hard to keep them lean. They’re raising token costs. They can make the agent worse as you keep adding and adding and adding stuff to them.

And the official guidance that I’ve read is that you should be periodically deleting your CLAUDE.md, your skills and your hooks, like every couple of months, rebuilding only what matters. But my own experience with that is that people have this fear of, hey, I don’t know if that’s going to actually make things significantly worse. I’m worried that if I do, that quality is going to drop very heavily. And I don’t have an easy way to just quickly restore things or try this out. There’s so many different configurations I can use models with. And so it can feel like this advice comes across as easy to try out when it’s not always going to be that way. And sometimes all of these markdown files and practices can become outdated and they can hurt more than they help. But I do think that there is something in this advice about how models and harnesses do actually get much better.

I’m a fan of skills. The critiques are still fair.

I’m personally a big fan of Agent Skills. I and a few of my friends maintain some Agent Skill packages that have gotten a little bit of traction. I maintain Agent Skills, which is an SDLC-focused pack. My friend Paul Bakaus maintains Impeccable, which is a design-focused pack. And sentiment has been generally pretty positive with developers using skills with coding agents. They’re seen as a good abstraction for turning a generalist into a specialist, right? And they work pretty well with a lot of different coding agents and harnesses. So people like them.

One of the sets of critiques has been fair. Of course, you have many different people writing them for very different domains. There isn’t really a great playbook for how to do this. So we’re all really playing it by ear. And we try to learn from each other. We take on feedback from the community and we’re constantly iterating on these things. But skill hygiene continues to be very important. Sometimes you’ll find skill packs that have thin descriptions, or they’re a little bit on the vaguer side for their workflows. And so I still think that agent skills are something that I believe in, and I think that they have a lot of value. But as people have really leaned into them over the last couple of months, auditing skills has also become increasingly important.

You could be doing a bunch of different parallel projects or parallel tasks on any given day. For some of them, maybe you’ll try out new community skills, or maybe you’ll try putting together your own skills for them. And if you can imagine, over the course of a couple of months, those skills locally can build up, and you’re probably not going to use all of those. You’re probably actually going to use a fraction of them.

When I’ve gone and I’ve read Hacker News discussion threads on public skills, the debate is very much, you know, there are people who find value in skills. There are people who call them net negative. There are people who feel like there isn’t enough evidence presented about the value. There are folks who feel like they just add a lot of noise. There’s heavy token costs. They’re unreliable. I certainly think that there’s a lot of valid feedback in here. It is sometimes challenging to come up with enough evidence to show, for everyone’s workflows, that these are actually a net positive. But there are plenty of people who say they find these things beneficial, and then plenty of cases where the feedback is valid. Like, hey, show me that this is actually going to be useful enough for me to consider for my project.

Why agent configuration rots

People keep adding rules to their CLAUDE.md files and their skills every time that they see their agent steering them in the wrong direction. I’ve certainly done that over time. And then the file balloons, adherence drops, you add more and more rules and quality can end up getting worse. And you almost end up treating it as this full knowledge base instead of a short decision guide. And that’s a very classic mistake.

AGENTS.md and CLAUDE.md bloat is also another big problem. I think it’s pretty widely acknowledged at this point. There have been a number of different research exercises done on real repos, and they found that there were a lot of configuration smells that were pretty common. Context bloat is pretty common, skill leakage, lint leakage. And most of the agent files that these research exercises have tried out had at least one issue. Files generally do grow past the Anthropic guidance of 200 lines. Some even reach hundreds or thousands of lines, wasting tokens on every session.

And I’m to blame as well for this. When I try looking at some of the CLAUDE.md files that I’ve put together in the past, not for sharing with people but just in my own setup, I’ve also gone past 200 lines. And I found that there were a few common failure modes that I’ve seen in my own files. Things like overly long examples. Redundant content that may have been present in readmes or package manifests or skill files. Add a rule every time the agent errors. That kind of growth can keep compounding. And so I think that you kind of have to think about it in a very, very focused way. And I’ve also seen that being over-specific in your CLAUDE.md file, in AGENTS.md, can sometimes not actually lead to the outcomes that you want.

For the numbers: a June study of 100 popular repositories found lint-related leakage in 62%, context bloat in 42%, and skill leakage in 35%. And in The new rules of context engineering, Anthropic says it removed more than 80% of Claude Code’s system prompt for its Claude 5 generation models with no measurable loss on internal coding evaluations. That result is not a target; the evaluations are not public, and it covers specific models in a specific harness. The lesson is that instruction value can expire, so archive first, and if a rule must always hold, encode it in a test, hook, or permission rather than leaving it as prose the model might lose.

I think there’s been some really good research I’ve been reading over the last couple of months, some good empirical studies that will maybe help with the discourse. And I wanted to make sure that I was covering some of this in an article for people, as I think some people haven’t had time to read some of these papers as well.

Do personalized skills help coding agents?

I’ve been wondering how useful it would be for Claude Code or Codex to gradually learn how I like to work. Maybe I prefer small changes, want tests run a certain way, or don’t want the agent refactoring unrelated code. This paper tries to turn that interaction history into a reusable personal skill. The surprising result was that personalization didn’t help very much. A skill based on one developer’s history performed about as well as a skill borrowed from somebody else. A generic skill built from lots of developers was more useful overall.

For many of us, we think that having a bunch of skills for our specific workflow can actually make a huge difference. But some of the research actually says that that’s not necessarily the case, and that having skills based on broader engineering, broader community best practices, can actually make more sense and actually lead to more value. And, I’m sorry, I should also say, where they can add value is if you have skills that include more examples for specific tasks. And sometimes those broader community skills will have this. For example, if I’m trying to tackle a problem related to, let’s say, scheduling, and there’s a bunch of different ways to approach scheduling, a bunch of quirks around it. If my skills, whatever ones I have, have a bunch of very concrete examples and specificity around it, maybe those can help guide the agent in a certain way. Otherwise, just having some details about, oh, I’d prefer the formatting of my scheduling primitives to look this way, that’s not actually all that helpful.

Personalization did look more promising when the same preference recurred across several similar tasks, though the experiments used an LLM-based developer simulator, so treat this as promising rather than final. My takeaway is to begin with a strong generic skill and add personal rules gradually. I wouldn’t promote a preference into permanent agent memory because I mentioned it once.

Skills once again, when I talk to companies and I’ve talked to enterprises, they see it as high value. They think that individual engineer skills are useful, but then when you have a set of skills for a team or an org, they think that that can lead to compounding value, which is kind of cool. So you’ve got your engineering culture captured in there, your compliance rules, your brand, your internal tooling quirks, how you want to approach consistency across people and different coding agents, how you want to approach the review process and provisioning and any of those types of things.

Do context files help coding agents?

I’ve been using files like AGENTS.md and CLAUDE.md as a kind of operating manual for coding agents. This paper asks whether those files actually help Claude Code and Codex solve more tasks. Across 288 runs on 17 real tasks, they didn’t make a clear difference to correctness.

Context files did change how the agents worked, though. In one repository, the guide warned that the full test suite was very slow. Claude responded by running more targeted tests and wasting less time. It didn’t become better at implementing the feature, but it followed the repository’s workflow more efficiently. I think that’s the useful distinction. A context file can tell an agent about expensive commands, generated files, architectural boundaries, or project-specific safety rules. It can’t necessarily teach the agent how to make a subtle design decision; the near misses usually came down to implementation judgment, and more repository prose wouldn’t have solved those problems.

My takeaway is to keep repository context files focused on things the model can’t easily infer from the code: how to run the right checks, which operations are expensive, what must remain untouched, and where the unusual project conventions live. I wouldn’t fill them with generic advice about writing clean code. A related study points the same way: prose summaries answered 4 of 45 behavioral questions about code while the source itself answered 27 of 45, because summaries smooth over the small details that matter. Point the agent at real code, not descriptions of it.

What I found when I audited my own setup

I was really happy to see Claude put out the doctor command in Claude Code. It’s a good hygiene command, and it basically runs a checkup covering unused skills and MCP servers and plugins relative to their context cost, whether you’ve got an over-specified CLAUDE.md file, slow hooks, or cruft, or things like that. And when I’ve run doctor on my own setup, I’ve been just shocked at things that were still hanging around that I’d completely forgotten about. Like, if you’d asked me, I wouldn’t have guessed that they were still there. I’d completely forgotten that I’d even experimented with them.

I’ll give you one example. At a point in time, maybe four or five months ago, I was curious about different writing skills that people were checking out. There were a lot of different anti-slop skills that people were experimenting with. And so I had tried out a bunch of different ones of those. And I was shocked because I’d completely forgotten how many of those I’d had installed. I had no idea. Like, are these things being triggered together? Or is one taking priority over another? Are they all being ignored? I’d just not realized that those things were still hanging around at all. And so auditing that was very useful. There were some design skills that some friends had written that I was trying out that I’d forgotten about. And now, as we’ve seen the community in some cases converge on some high-quality skills, I would prefer to lean on those than some of the other experimental ones that I tried from a few months ago.

I remember recently seeing a tweet where somebody was saying, yeah, I started auditing my skills and I went down from 250 to 25. And I was just like, how do you end up with 250 skills? That’s crazy. But experimenting with a lot of community skills, these can easily compound over time. And you don’t want to confuse your agent, right? So you’ve got to periodically lint and clean up your agent environment.

Installing a useful skill and keeping it forever are separate decisions.

Auditing quality, not just quantity

I think another thing about skills is just making sure that they’re high quality. Anthropic’s own Skill Creator now includes evals and a benchmark mode for trying to check on quality. There are good community tools for checking on SKILL.md quality. Like, do they have good descriptions, good triggers, clear steps and examples? There are more security-focused auditors and best practices for skills as well.

One naming trap worth knowing: in Claude Code, /doctor inside a session is the configuration audit, while claude doctor in a shell only prints installation diagnostics, which is why some people report that doctor “only shows a status check.” And an installed skill does not dump its whole body into every prompt: names and descriptions load for discovery within a listing budget defaulting to 1% of the context window, and the body loads on invocation. I also review memory separately with /memory, since auto-memory can hold stale preferences even after the project files are tidy.

Audit on a cadence, then test removal

And so I think there’s high value personally in, like maybe every couple of weeks, maybe at once a month even, just running doctor on your skills, on your setup, and auditing what you’re doing and seeing, okay, well, what still holds true? And then if you have the time, actually going and seeing, like, if you were to delete your skills, or you were to instruct your agent, well, don’t use any local skills at all. Only try to complete this task using the raw model and harness. And see, is it actually okay without using any of these skills? And then you can ask yourself, okay, well, actually, maybe it’s okay for me to delete these skills. Maybe that’s fine.

But I feel like sometimes we lean on skills and all of these things we’ve installed as a crutch, because we feel like it’s unsafe to remove them, because we don’t trust that the model and harness have actually gotten better. So I think that there’s a lot that we can experiment with and learn.

Continue to have hygiene around them. Audit the quality, the security. Do run doctor commands regularly, and try to just make sure that you’re keeping your local setup as lean, but as specific, as is needed.

Thanks once again to our sponsor Oracle. Do check out their Agent Loop Decoded post which is a pretty solid read the three levels every agent engineer must know.

If so:

• You’ll likely need humans in the loop upfront for deciding on product intent, system design (if you care) and your quality bar.

• Do review code (lights-on factory) but be intentional with where it’s needed the most. I’ve found you want to watch out for where automated back-pressure breaks. Or where maintainability trade-offs need to be made.

• Aim for quality checks to happen as early and continuously as possible. Not all of them have to, but this includes type systems, automated tests, mutation testing, security scanners and linting for architecture rules.

• Number of checks != quality. You’ll likely need to experiment with what checks give you the best signal to noise ratio. Be ready to tighten or relax your constraints deliberately.

You want to build your factory so some aspects of human taste get encoded in the environment, the agent gives you evidence of its work being right and where a human still “owns” what ships to production.

Do you really need a software factory?

In my experience, you can get surprisingly far with your stock coding harness! i.e. Claude Code or Codex, multiple sessions, good SPECs with verification baked in and constraints. You can even throw a batch of GitHub issues at them with implementation and human-involvement criteria, but it’s when this system needs to be repeatable and event-driven that a factory is helpful.

So I started off by saying a software factory is a repeatable loop around software work. We can actually look at a prompt that demonstrates a very small factory loop here:

Read GitHub issue #123 and the repository instructions before changing code.

Implement only the stated acceptance criteria. Do not modify authentication,
billing, migrations, or existing test assertions. Work in a branch and keep the
diff reviewable.

Run npm run lint, npm test, and npm run build. If a required check cannot run,
stop and explain why. Open a draft pull request with the checks you ran, the
remaining risks, and any decision a human still needs to make. Do not merge.

A goal can keep this moving until the checks pass and we can poll GitHub issues for any specific labels or review open pull requests each morning. Branch protection could enforce a merge boundary and the human can stay in the loop by choosing what becomes ready, reviewing and making the final merge calls etc.

Add a software factory when you need an event-driven queue of work (e.g. Slack triggers, GitHub issues, Linear, a backlog) to run in an isolated cloud environment to handle triage, implementation and testing with some explicit human babysitting. Some end their loop with a monitor agent watching production and filing issues which triage again.

In my experience, the factory becomes useful when the hard part is making your different runs behave consistently, handing work off between agents and avoiding different sessions from claiming the same issue, preserving evidence and stopping production when human review is falling behind.

What solves this might sound a little boring. For example, Warp mention triaging every incoming issue into one of four states - ready-to-implement, ready-to-spec, needs-info, wait-to-implement - and the label is what fires the next agent.

This label does a few jobs in one go: it’s the queue, the lock and since a session only picks up what’s marked ready, it’s where a human can park stuff without saying no permanently.

Workflow wise, there are a few similarities and differences to just using Claude/Codex:

• Steering: agent needs course-correction, you can give input and redirect it

• Notifications: how the factory says its blocked. This can be because a requirement was ambiguous, it started something risky or it needs human input (steering)

• Handoff: move the task, its state and context between the cloud factory/another agent/human reviewer. Good handoffs will keep track of what happened, whats left to be done and why the handoff is needed.

In a good factory, the human isn’t limited to just reviewing and approving the final diff at the very end. They can shape the work early on, steer it during implementation, get it through a handoff or stop it shipping to production.

Verification is where a responsible factory spends a lot of its time. We’ll cover this more later.

If you decide you do need a software factory, building it isn't the only option. Standing up the infra to scale a factory can be a lot of work and you may want to consider buying vs. building. Factory, Warp and HumanLayer are all working on this.

What my day looks like now

My day-to-day experience of software development has changed a lot over the past year. I’ve been talking about increasingly doing a lot of parallel work with agents, moving towards having a lights-on software factory. And a lot of people have been asking me, like, what do these things actually mean? What are you building? What are the kinds of projects that you’re using these things on? It’s a lot of this:

So on a very average day, I have a simpler lights-on software factory. I can have tasks that are running in the cloud. Half of these tasks might be working on production client applications with smaller companies that I’m working with. They’re going to have real users. They’re going to have real authentication, payments, subscriptions, real beefy risks that you need to be careful with. You can’t just say, oh, agent, just go and do this stuff without having tests and constraints and quality checks in place.

I could work on my open source projects. I could be building out companion sites for my books. I could be working on tools. I could be building apps of my own. And these are all very, very different kinds of applications that I’m working on. And sometimes all they have in common is the tool that I’m using to work on them, right? Maybe I’m working on a migration. Others, I might be doing actual beefy feature work. And the blast radius of the work might also be very, very different.

So as you begin to think about getting to a place where we’re increasingly doing a lot of parallel work, we’re trying to improve velocity, we’re trying to improve productivity, and we’re trying to improve autonomy, which means getting the system to a place where we trust it more, you do have to think about what are the places that absolutely require human code review, human input.

And a lot of that is going to be required up front, right? When you’re defining your specification, your requirements, what is the design of the product going to look like? What is the intent of the product going to look like? And then how are you verifying that the agents have actually gotten the work done right? How are you verifying that they haven’t broken the existing system that has been in place? How are you making sure that it’s meeting your quality bar?

And so generating code is not necessarily the part that you need to worry about the most. Given enough context, agents can write the implementation and run the tests and inspect failure and revise code for us. We need to get to a place where we feel like there is enough of human taste encoded in the environment that we can trust what is being built, so that our human attention can be focused on the places where it’s needed most.

Now, there is pushback from folks saying, hey, well, I don’t buy that you can just automate away a lot of this stuff. It’s not to say that we’re automating away all of it, right? But given the volume of code that’s being generated, I don’t think that it’s realistic for humans to be reading all of it, especially when we’re not building rockets a lot of the time, right? We’re building UI, we’re building full stack applications.

Our judgment, our taste is best focused on the places where it’s needed the most. Like, what are the riskiest parts of the systems? Where do we need to apply human taste? And that can be in the front end. That can be in how the system works. It doesn’t have to be 100% of it.

My cognitive bandwidth does not scale with the agents

The reality is, yes, we can now fire up dozens, hundreds, thousands of agents in parallel, but your own cognitive bandwidth does not scale in the same way. This can feed into cognitive or comprehension debt which I’ve talked about before.

If you remember back to just five, ten years ago, there was a lot of discussion in the engineering community about context switching and the cost of it. We would talk about how people hated when a colleague or someone would walk up to your desk when you were in the middle of a task. It would then take you so long to get back into your flow state because you had to catch back up in terms of like, where was I? What was I doing? Even if you had a little bit of residue there, it still took you time.

We’re now context switching even more than we did before. On any given day, if I’m working outside of a software factory, I can be working on five or ten different projects with agents at a single time, or five or ten different features on a single project at a time. I can have five or ten different sessions, you can effectively say.

That means that I have to be able to stay on top of at least a few of those. It is possible that I’m going to be able to increase how much autonomy I give some tasks if I have trust that I’ve defined the task well enough, I’ve defined the outcome, how it’s going to verify that it’s done well enough. But then there are going to be tasks where maybe I don’t necessarily feel that way and there’s more risk involved or more nuance. I’m going to have to pay attention.

Consider optimising the software factory for your reviewer. Given every one of those approaches still routes its output to one person's attention, you should ask how much cheaper the factory is making the decisions you still have to make.

A wrong-project mistake

I remember when I’ve been working on multiple parallel projects with my agents, and there have been times when I’ve accidentally done things like, maybe I was working on a web app where I wanted to add in a dark mode, and so I had in my head, okay, well, this is what the shape of this needs to look like. But I accidentally went to the session for a different project, and I started putting in that same prompt.

So I began implementing dark mode for something that absolutely didn’t need it. And so I can make that mistake. I don’t want my software factory making that kind of mistake.

You need to think about this really in terms of a system. You are effectively trying to encode a software engineering culture, a team culture, into a system so that it has those same kinds of behaviors, so that it has ownership that belongs somewhere, so that someone is still on the hook for what happens, and you’re being very explicit about how you think about those things.

When green is misleading

Even in these systems, you want to be very careful, right? Many of us have seen that when you have asked AI to help you pass a test, like we’re talking about a programming test, a unit test, it can change the unit test to satisfy that condition, or it can change the logic of the code to pass that condition. That doesn’t mean that it’s actually followed your intent in order to align both the functional behavior and what the test was supposed to be testing, right?

Just because a software factory is showing that everything is green doesn’t mean that it’s actually green, especially at the start when you’re setting these things up. You need to pay a lot of attention to make sure that your checks, your verifications, all of those are shaped the right way. They’re doing what you expect them to be doing. You don’t want them to be misleading.

You don’t want a situation where you had tests that said, hey, actually, I have gone and changed what authentication providers are supported. You asked me to add GitHub for example, as an authentication provider, but hey, my UI only had space for three, so I’ve gone and I’ve dropped one of the other ones. And hey, by the way, that happened to be one that your customers actually wanted. So you just need to be very explicit about how you want these systems to work.

Btw security is super important too and if your factory reads untrusted input like a GitHub issue/Slack message it might be adversarial and include problems like supply chain attacks. So some explorations into software factories, like Vercel, run their agents in isolated sandboxes holding just the secrets a task needs. That way a compromised run can’t reach what the job doesn’t need. Your defense ends up being layered.

Which old projects deserve another life?

I also think that a big part of how we work these days is deciding what should exist. If you remember back to many years ago before AI, there were so many abandoned software engineering projects, so many abandoned weekend projects, personal projects where they just wouldn’t launch because we didn’t have the time to finish them. We didn’t have the bandwidth to prioritize getting them out the door because they just weren’t that important to us or we couldn’t find the time.

Now it’s fairly trivial for us to complete those projects, but the same human judgment question comes in. Do those projects deserve to exist? Should they be launched? Because you put them out into the world and even if it has just five users, maybe you have to maintain it. Maybe you have a quality bar now that you want to maintain.

I know that I’ve had so many GitHub projects from over the years where now that I have an agent, the first thing I do is get the thing building. Because, of course, you clone it and now it doesn’t build because all the dependencies have changed. Half the things are out of date or have security vulnerabilities all over them, so you have to update that.

Then you have to add tests if you didn’t have tests so that you know that behavior is at least going to be there if you’re upgrading the project in some way, or if you’re migrating it to a more modern language or framework or thing like that.

Then you start to ask yourself, well, maybe, a silly example, but maybe I used Twitter Bootstrap back in the day for this, but now everybody is using Tailwind and shadcn, so I have to re-implement the UI. And what you’ll notice is that suddenly this is taking you more time, right? Yes, the agent can get a lot of this done quicker, but you’re now having to factor in product sense and taste and all of these things.

You still question, well, who is this for? Does it have a market? Is it for myself? Is it for other people? If I’m putting it out into the world, is it still going to be as interesting given that now anybody can spin these things up as quickly?

So I think that human question of do these things deserve to exist? How do we factor in our taste and judgment? I feel like those things continue to be extremely important. That’s where that scarce resource of human attention still really comes in. Back in the day, we only had a finite number of hours in the day. We had meetings. We had to budget in time for design and coding and so on.

Now that we have agents to help us, I think that you have to really just be very explicit about where you’re spending your time and why.

What happened when I built a sample one

So I’m going to talk about the 82-minute factory run. People have been asking me for quite some time, you know, “How do I build a software factory?” Or, “I’m used to using Claude Code or Codex. How do I evolve my setup to using a software factory?”

So the first thing that I’ve been saying is, “You may be fine. Your work may actually be totally fine without needing a factory.” But I did want to give people a reference setup that they can check out. So what I put together is a repository called Factory that you can go and check out. I also put together a demo application and workshop.

Now, for the last couple of years, my go-to demo application for a lot of things has been a movies app. I’m a big movies fan. I love watching movies. I watch movies all the time, and so I have a demo application, which really starts off as a very simple movies app. And what I want the factory to be able to do is go ahead and implement a number of features. There’s a few different features. I want a favorites feature. I want it to be able to maybe do search, and maybe also want a dark theme in there as well, those types of things.

So I have my factory go and begin working with these things. You can check out the implementation. One of the benefits of it was actually catching real problems. These problems may not have been things that I would have caught if I had just asked it to do a one-shot implementation.

Maybe around the 60-minute point, I was feeling like, “Wow, this is going unusually slow.” I asked my harness using the factory, “Why are things going slow?” It said, “This is actually totally fine. All the verifiers are still running.”

You might have expected individual tasks to take 10 minutes, 15 minutes, 20 minutes, but they can take two to four times as long once you begin to include verification, retries, browser checks, human review, any of those extra delays.

I do think that these can add up to better quality and better trust in the system. From a measurement perspective, you might look at metrics like cost per merged PR and code shelf life as comprehension debt metrics.

You also need to think about what is useful delay versus factory overhead. The verifiers, in my case, caught some real problems. A little bit of the time was maybe sunk into producing evidence that I wanted. Some of it was overhead in the factory running. I didn’t really spend any time optimizing it, but a factory that just runs a lot of checks that you’re not finding valuable does not mean it’s a high quality one.

You want to study how, for any repeated checks, are they irrelevant? Are they noisy? Are they actually making the system safer?

A verification budget

The way that I think about the budget for verification, this is basically what we’re talking about. We’re talking about a verification budget. I think about it in the same way as I’ve historically thought about performance budgets.

There are going to be certain kinds of checks that you can run early on in your software development lifecycle, and there are going to be some things that are so heavy, but they offer so much value that you will want to run them later on. There are some kinds of fast checks, linting, for example, type checking. These are relatively fast checks that you can run early on.

Our full suite of tests can be run closer to right before a draft PR is being put together or after that. That can include mutation testing, browser testing, security checks, anything like that.

I think that you don’t necessarily want to replace these with just summaries. You want real tests, but you just need to make sure that you’re budgeting for them in the right places because you don’t want to slow down your development loop. I certainly never want to slow down my development loop. Having a fast iteration loop is important to me, but I also want to still have those checks and balances.

When a run doesn’t ship

A lot of what I’ve written above concerns the checks.

In their software factory, Vercel marks every agent run as “success”, “flawed”, “blocked” or “manual” and only “success” ships to production. The rest re-enter the system. I’ve been thinking about runs in similar terms.

“Flawed” here means the wrong thing was implemented or maybe it didn’t have full context, so that has to be fixed. Blocked means the environment may have been missing a credential so you have to provide it. Manual is a boundary the factory may not be allowed to cross it yet.

Two of the three things here may have mechanical fixes and the last one is about trust.

While this is great what sorting doesn't show you is cost. B

ack to my factory implementation with the TMDB app, the quick finder with no rejections took 7 minutes. Favorites, with two rejections and a human decision in the middle, took 56. Same factory. So I'd pair the taxonomy with per-stage timing, otherwise you know a run came back flawed without knowing what finding out cost you. The other thing I'd fix is the handoff at the boundary: my sample factory stopped issue the first issue and moved it to factory:needs-info, which was right, but I didn't know where to put my answer. A manual run isn't finished when the factory stops but when the human knows what to do next.

Autonomy is not a single setting

I wrote a couple of weeks ago an article about agentic autonomy and how to think about autonomy because autonomy is not going to be a single setting for every single project.

Verification buys you trust, and it buys the ability to grant more autonomy to your agents. So if, for example, I am working on a non-trivial change, but I have a number of checks in place, everything gets verified correctly, and maybe I’ve hand checked it myself. The next time I’m going to do a task like that in the same project, maybe I’ll feel comfortable giving the agent a little bit more autonomy.

That’s the thing that you think about when you’re building these software factories. Your verification is going to change with risk. Your goal is the best signal to noise ratio. You don’t just want to have some large checklist that you’re running.

The feature I had to relearn

There was a feature that I’ve been putting off on a day when I’ve been using multiple sessions with Claude, and I was working on a few different projects at a time, a few different features at a time per project. And so Claude had implemented the feature that I was working on. It looked like the tests were passing. I hadn’t put a lot of thought into verification, but the tests passed, and so I thought it worked. I merged it.

And so this was a favoriting feature. I thought that this was actually pretty good. I tried to check it out in the browser. It seemed like it was okay, but a couple of days later, I actually returned to the code because there were some tweaks that I thought I might make to this.

I didn’t want to just ask my agent to make the changes, because it was just a subtle way that it worked. You tap on the icon, and it would not show the right effect on tap, and so I wanted to just tweak it. I wanted to understand how it worked so I could guide my agent correctly.

I returned to the code, and I couldn’t explain to you how the feature worked. This repository was mine, right? I’d approved the change. I understood how a lot of it worked, a lot of the repo worked, but my understanding hadn’t kept up pace with all of the code that had been building up.

What I failed to absorb was how this feature that had been added actually worked, how the UI worked, how the effect on it worked. I had to redo this feature and actually go step-by-step, “How does this work? How can I understand it?”

What parallel work does to understanding

When you’re doing parallel work, it amplifies this overall problem, and it gets even more amplified when you’re doing it in a software factory. When you’re doing five or 10 sessions, they create much more than just a review volume problem. They create several mental models that can end up going pretty cold while you’re working elsewhere.

We’ve historically talked about the challenges with context switching, and as soon as chat compacts, you reject some approaches, you try out different things, you’re pairing with the agent, you’re going to have a difficult time remembering everything that happened in your session.

You can scroll up, and as compaction has been happening, you’re not going to have everything there, and you’re not going to be able to store it all in your head. Code often preserves a decision that was made, but not why the decision was made.

This is something that I think can be a useful learning for you, where it’s important, consider asking your agent to actually store information about its trajectory, or interesting lessons about how it approached a problem so that you can go back to it later.

This can or can’t be something that you decide to commit to a repo. You can keep it local if you want, you can share it with a team if you want, but that can be something that can then be consulted later on. Rather than you relying on it maybe being in a session, or you maybe remembering about it later.

Ownership doesn’t disappear

There is a broader principle underneath all of this.

The percentage of code physically typed by humans may fall dramatically. I don’t think human ownership needs to fall with it.

• Someone still chooses the problem.

• Someone still chooses the architecture.

• Someone still sets the quality bar.

• Someone decides which verification signals deserve trust.

• Someone decides when the evidence is sufficient to ship.

And when the resulting system fails, “the agent wrote it” doesn’t cut it. This is why I don’t think the future of software engineering is best described as humans leaving the loop. Instead, human judgment is being relocated.

We should remove people from the parts of the loop where machines can produce stronger, faster, more deterministic signals. At the same time, we should concentrate people around the places where context, taste, risk, and long-term ownership matter most.

The best software factories will not be defined by how completely they eliminate human involvement.

They will be defined by how intelligently they place it.

Keep human judgment upstream on intent, system shape, and the quality bar. Review code where automated back-pressure becomes weak or the consequences become subjective. Push every deterministic signal as early and continuously into the loop as possible. Tighten and relax constraints deliberately as the system earns or loses trust.

A human still has to own what code ultimately ships. Code good enough to ship still starts there.

Now within that, you’ve probably heard about loop engineering. I talked about it a couple of months ago when I wrote a big blog post about it.

A loop is an autonomous, self-correcting feedback cycle where an AI agent repeatedly acts, tests its results and adjusts its approach until a specific goal is met

A quick message from our sponsor:

There are now basically two core primitives you can think about. In Claude Code you have a goal primitive, which can drive a single bounded task forward until you’ve got a particular goal, like a measurable finish line that’s been met. And then loop reruns on a timer or a fixed interval, so you can use it to kind of schedule changes.

Before the primitives were primitives

I remember back before we had primitives baked into Claude Code and Codex, loop engineering was heavily about setting up your own bash loop, a hand-rolled thing. That’s how I approached it. And you might remember earlier in the year, a number of us were playing around with the Ralph loop by Geoff Huntley. We were experimenting, we were sharing our workflows, we were sharing what worked and what didn’t, but it was largely on some of our personal projects where, if we ran into a wall, it didn’t really have a big cost to us because it was on personal projects.

And as we’ve tried to see what patterns, what aspects of loop engineering are now a little bit more baked, I think we have a clearer idea of how you babysit it. I can now largely rely on the output of the primitives in Claude Code and Codex. We’ve come a little way. But at the same time, you need to be very diligent, because loop engineering where you kind of leave it sitting alone, and you haven’t really thought about whether the end goal or the constraints have been well defined, can leave you in a problematic state. This is why there’s nuance when deciding to use it for an evergreen codebase without users or as much historical complexity vs. say a brownfield bank codebase.

How the Claude Code team frames loops

The Claude Code team published their take on four kinds of loops and it lines up with how I use the primitives (their write-up). Before we get to it, here’s my quick summary:

On the Claude Code team, we define loops as agents repeating cycles of work until a stop condition is met. We categorize a few different types of loops based on: how they are triggered, how they are stopped, what Claude Code primitive is used, what type of task is most appropriate for each. Not all tasks require complex loops; start with the simplest solution and use these patterns selectively.

Every prompt you send starts a manual loop with you directing each turn. Claude gathers context, takes action, checks its work, repeats if needed, and responds. We call this the agentic loop. For example, ask Claude to create a like button. It reads your code, makes the edit, runs the tests, and hands back something it believes works. You then manually check the work, and write the next prompt.

Their write-up walks each rung.

On goal-based loops:

Sometimes, a single turn is not enough, especially for more complex tasks. Agents do better when they can iterate. You can extend how long Claude keeps iterating by defining what done looks like with /goal. When you define the success criteria, Claude doesn’t have to make a determination on what is “good enough” and end the loop early. Each time Claude tries to stop, an evaluator model checks your condition and sends it back to work until the goal is met or a number of turns you define is reached. This is why deterministic criteria, such as number of tests passed or clearing a certain score threshold, are so effective. For example: /goal get the homepage Lighthouse score to 90 or above, stop after 5 tries.

On time-based loops:

Some agentic work is recurring: the task stays the same and only the inputs change. For example, summarizing Slack messages every morning. Other work depends on external systems, and a simple way to interface with one is to check it on an interval and react to what changed. For example, a PR which may receive code reviews or fail CI. For these, you can trigger when Claude runs with /loop, which re-runs a prompt on an interval. For example: /loop 5m check my PR, address review comments, and fix failing CI. /loop runs on your computer, so if you turn it off, it stops. You can move the loop to the cloud by creating a routine with /schedule.

And on proactive loops, the top rung:

Triggered by: an event or schedule, with no human in real time. Stop criteria: each task exits when its goal is met. The routine itself runs until you turn it off. Best used for: recurring streams of well-defined work: bug reports, issue triage, migrations, dependency upgrades. Managed usage by: routing routines to smaller, faster models and using the most capable model for judgment calls.

Their verification advice is worth lifting whole, because it moves the manual checking into something Claude applies itself:

---
name: verify-frontend-change
description: Verify any UI change end-to-end before declaring it done.
---
# Verifying frontend changes
Never report a UI change as complete based on a successful edit alone.
Verify it the way a human reviewer would:
1. Start the dev server and open the edited page in the browser.
2. Interact with the change directly. For a new control (button, input,
   toggle): click it, confirm the expected state change, and screenshot
   before/after.
3. Check the browser console: zero new errors or warnings.
4. Use the Chrome Devtools MCP, run a performance trace and audit
   Core Web Vitals.
If any step fails, fix the issue and rerun from step 1 - do not hand
back partially verified work.

Goal

The way that I use goal is I use it for building any specific piece of work until it’s provably done. Like for example, you can use a goal to say: make sure that this experience loads in five seconds, keep going until it’s done. And it will continue to use an independent evaluation check to keep checking if the completion criteria has been met. What’s better is to be even more specific about what tooling is being used.

As for real goals that I’ve run to a number, there’s a few things that I’ve done. I’ve used goals for running through GitHub issues: review and close the last 10 issues, or review and move the last 10 issues forward, something like that. And that’s semi-open-ended, right? Or: let’s make this page load 50% faster. Sometimes that works well, sometimes it doesn’t, but it’s really about the experimentation.

Loop

Loop is a little bit more of a scheduler, so it keeps an eye on something or repeatedly executes a pattern on some cadence. So think of it a little bit like a cron. It’s best for doing things like polling logs or monitoring external states. You could use it for potentially checking in. If there are repetitive tasks you find yourself doing on some cadence, loop is pretty good for that.

What I delegate, and what I watch

For me, I use probably between five and ten agents every day. Very typically I’ll max out at about five concurrently. And some of those tasks might be ones that are a little bit safer. So maybe it’s, hey, I implemented this feature, go write the documentation for it. Or go double check that we have sufficient test coverage. Or something in that vein. If I’m working on a more complex problem, or something where I know that even if I’ve given it a good spec, or what I think was a good spec, and I’ve tried to give it some stopping conditions, there is still a reasonable chance it may not get everything right, I would try to watch it more closely. If the task involves anything just a little bit sensitive, whether it is I’ve given this access to a system, or whether the feature happens to touch authentication, or something related to security or finance, I’ll definitely be watching that closely.

Generally speaking, I do think we’re going to get to a place where folks are increasingly comfortable with delegation, as long as they’re able to have these clear ways of verifying that their goal or their stopping condition has been met. But you do still need to take a look at the code, at the thing that’s being generated, to make sure that it’s meeting your mark.

The other habit that matters here is not letting the agent that did the work decide the work is good. One sub-agent drafts the change. A separate one verifies it.

Sometimes an agent can be confident about something, and a verifying agent can catch things that they weren’t necessarily expecting. Like if it thinks that the baseline performance of an experience it’s generated is actually fine, and it’s only evaluating performance based on desktop, but you’re actually caring about the experience on mobile. That might mean the agent is being very confident about one dimension of the problem, but not the other.

And I learned this one the hard way. I was curious to figure out: is there something that we’re missing that we haven’t gotten direct feedback on from our users, either in an issue tracker or in a comment somewhere? And so I asked it to go and take a look at some of our competitors, and put together a list, and some PRs, not pushed, but some PRs locally, of what solving some of those gaps might look like. And I almost pushed some of those changes. But I didn’t actually look at them closely enough. I read through its research, but I didn’t look at the implementations closely enough. So I delegated the task, but I was close to delegating the judgment as well. Now, when I actually looked through the changes, what I realized is that it would introduce a lot of additional complexity for our users, for, I think personally, not all that much gain. And so I feel like you need to sometimes check yourself, that you are not delegating the taste and the judgment to your agent. You’re delegating the task, and then you are actually checking back that it’s meeting your bar.

The evaluator sitting behind goal is not that checker, by the way. It doesn’t look at the content to see if it’s good or bad in any way, shape, or form. All it does is examine the conversation transcript to see if the hard rules you specified have been met.

/goal Refactor the data-fetching layer in Dashboard.tsx until Lighthouse performance score is >= 92 and LCP is under 1.8s as shown by the Lighthouse CLI output. Do not change the public API of any hooks. Each turn must improve at least one reported metric; abort if two consecutive turns show no improvement. Stop after 10 turns.

The workflow I run every day

One of the workflows that I do every day: I have a popular open source repository called Agent Skills. We’ve got over 80,000 stars, and up until recently we were getting anywhere up to like 80 or 90 pull requests that we had to review a day. And so every day I would go and I would spend some time checking on this. Now with loop, what you can do is say: well, every 24 hours or every 12 hours, check the GitHub repository for any new open issues and provide a summary of their urgency, or provide a first pass review, or anything like that.

/loop every 1h "Check the GitHub repository for any new open issues. Provide a bulleted summary of their urgency."

Combining loops and goals

Now you can also sort of combine loops and goals. So you use loop to schedule a check, and then goal to solve the problem. So you can say something like: loop for every 24 hours, check GitHub for issues labeled bug. If one exists, use goal to implement a fix until all local tests pass and push the branch.

/loop every 24h "Check GitHub for issues labeled 'bug'. If one exists, use /goal to implement a fix until all local tests pass and push the branch."

But also keep in mind that goal has got some limitations around just how much you can sort of cram in there.

Their write-up also has a composed example that shows where this is all heading:

The primitives above, along with other Claude Code features like auto mode and dynamic workflows (research preview) can be composed into a loop for long-running work. For example, to handle incoming feedback, you can use: /schedule (research preview) to run a routine that checks for new reports, /goal to define what done looks like, and skills to document how to verify it. Dynamic workflows to orchestrate agents that triage each report, fix it, and review the fix. Auto mode so the routine runs without stopping to ask for permission. Putting it together, a prompt could look like this: /schedule every hour: check the project-feedback channel for bug reports. /goal: don’t stop until every report found this run is triaged, actioned, and responded to. When fixing a bug, use a workflow to explore three solutions in parallel worktrees and have a judge adversarially review them.

What the triage system actually does

Inside the PR triage, when I have loops and goals working together for me, what I end up having is a system that allows me to really continue accepting PRs and issues, being able to stay on top of what is coming on my plate every day, and especially being able to cross-reference. That’s been a really big deal for me. If I want to work on a particular goal of, hey, we’re going to be redoing this part of the system, and I need to make sure that any issues that happen to touch it are closed as a result of this rework, or we’re not stepping on anyone else’s toes, I can make sure that that’s well defined.

Scheduled tasks are really useful for regularly just reviewing new PRs and closing things that clearly don’t fit. One good stopping condition, for example: we have a set of contribution guidelines, and our contribution guidelines include things like, hey, we currently don’t accept translations.

And it’s not that we don’t care, it’s that they’re difficult for us to maintain, because we don’t speak all the languages often coming in. And so if we tell it, close any issues or close any PRs which happen to touch that aspect of our contribution guidelines, that’s something that can do really well when it’s on that schedule. And so it reduces the overall batch of things that we need to review.

What loops don’t buy you

I’m often asked what loop engineering is not a great fit for. Generally speaking if you don’t have a clear idea of what the end-state/done/good means for your completion, it may not be the right pattern for your work. For example, a vague goal would be “keep going until this UI design is good”. What does that mean? Good to who? How is it being evaluated? Tasks that require human taste, subjective design, or open-ended creative exploration aren’t a good fit. When you have a pretty clear idea of the goal, I think loops are a good option to consider.

A final message from our sponsor:

The fine print

One classic sign that you’ve got a loop spinning in place is the same command being tried over and over without any change in the result. Give the same command a third time with no change from the second and it’s probably time to stop.

One bit of fine print worth knowing. Recurring loops expire seven days after creation. I’d been telling people this was three days. It’s seven. And loops are session-scoped, so they stop when you start a new conversation - though resuming that session with --resume or --continue brings back any recurring task still inside its seven-day window. If you need something that outlives your session, /schedule runs it in the cloud.

If there’s a check you already run every morning by hand, that’s your first loop. Mine was the pull request pile.

Software quality now depends on the constraints you set around your agents.

Constraints define what the system is allowed to do by throwing tests and deterministic constraints at an agent’s proposals. It’s by setting and maintaining these constraints that we build loops that reliably deliver high-quality production software, even when agents are creating hundreds of thousands or millions of changes every single day.

We call these constraints quality gates, and they take many forms. They include conventional unit tests, property tests, and acceptance tests. They include mutation testing, where we generate variations of code, run it against the same tests, and make sure that people aren’t sneaking bugs in that we’re missing. They’re metrics around code quality, such as cyclomatic complexity and line length, that help keep things readable. Constraints also play an important role in what proposals the system will accept and apply as code changes. By the time a change proposal moves from the interpreter running the agent to the agent controller and out to production, we’ve done enough checks on it to be confident that it’s safe to ship and the impact of its change is well within the scope of the agent.

An agent can propose anything. Your constraints decide whether a proposal is safe enough, correct, scoped, and useful, for you and your team to ship.

This model offers a lot, but also leaves out many pieces, and those omissions are worth thinking about today. One issue is autonomy; agents might apply their intentions well, but may fail when there’s missing information or when what they try to do is ambiguous. That applies both to the task itself and to how it’s parameterized by the harness, environment, and other components. Many of the reasons that humans fail to ship great code are shared with what agents might do: brittle environments that don’t hold up under script-driven stress, nondeterministic builds, missing permissions, and weak tests. This motivates a better environment that gives agents trustworthy feedback, allows for low-damage failure modes, and makes it easier to progressively build up success.

The environment we’re after is one where an agent can do real work, get feedback it can trust, and fail without doing much damage.

The other important issue is trust. We can’t credulously hand off intent to something even as smart and robust as a modern agent without checking for correctness. We start with trust, but it has to be hard-earned.

Some constraints shape work before it begins. Others give feedback while the agent is working. Others decide whether its output can cross the production boundary at all.

There are a bunch of ways to model how we put a verification structure around a system.

In my experience it helps to have a broader, but intentionally chosen, set of checks for your constraints instead of solely relying on unit tests. The idea is that each check has a distinct responsibility and that can range from type safety and performance to late-stage security scanning. Folks can define their own constraints too, including architecture rules that linting tools like ESLint can enforce. Many of these tools have built-in hooks that can be used to pull in agents, or humans, when things break.

For now, much of the difference between useful agent output and slop still comes down to the skill of the team operating the loop.

AI gives us high volume code generation and velocity, but this can also mean it gets harder for humans to review every single change. You have to instead be intentional with where their attention is going. If you put a human check into a system that otherwise moves at machine speed, don’t be surprised if that impacts productivity. Human attention is scarce and valuable so we should proactively direct it to those most nuanced problems that require our judgment. Downstream humans should only be pulled in when the automated guardrails for constraints break.

Human “code review” in the future is going to look very different

Correctness is one important dimension, but you may care about others as well, like maintainability, performance, security, efficiency, and comprehensibility. Just as correctness decomposes into many signal types, so does the rest of quality. And while it matters how many constraints we have in place, it matters more whether they’re challenging enough to meet our bar for quality and production readiness.

Software quality isn’t a single metric. Think of it as a collection of signals of varying importance to you and your team.

Back-pressure can be implemented through many tools: compilers rejecting invalid code, tests failing, security policies blocking bad practices, CI declining to deploy. Ideally it exists throughout the loop, not as a single review at the very end of all the work.

Constraints and back-pressure let agents catch bad work before its a problem

What happens if we can’t apply the constraint because the volume of changes is higher than our tools can consume? We end up building a queue and relying on a verification system that moves at human speed. To scale, we want to push as much as we can into the verification loop throughout and not wait until the end. If we can scale within our automated checks, we can increase the speed and throughput of our entire delivery system. If we run out of room in the verification loop, we need to do one of several things. First, we can scale our verification system and create more capacity to constrain and push back on changes that come in. Second, we can reduce the rate at which agents generate new changes so that verification can catch up to the volume of work. Third, we can lower our quality bar so that verification doesn’t push back as hard as it otherwise might. From a scaling perspective, we need to be ready to do all of these things. At the same time, we should not stop short of realizing that we could actually get more done by un-constraining in some directions. Maybe we can increase the speed of agent-generated changes by providing swarms of agent developers or automated software factories to create changes without waiting for us to review each of them.

And in some places we might want to give them more freedom as long as we keep tighter constraints in others. By providing tighter constraints where we care the most, we can maximize our throughput without sacrificing quality. Throughout these decisions there are many options. Most obviously, we have to trade between different dimensions of quality. As we’ve emphasized, security is very important, but we’ve also had to trade between delivering security and delivering a product on time. There is a spectrum from innovation-focused at one end to quality-focused at the other. Somewhere along the way, we have to make choices about where we want to be on that spectrum.

We want to send clear feedback from the environment and the system back to our agents or teams, so that people can focus on the more subjective concerns of taste, intent, and architecture. If we can help humans stay inside the safe range of constraints, we can avoid the need for them to work hard trying to figure out where things went wrong.

Software quality includes more than just correctness. Software quality also means maintainability, good performance, security, efficiency, and being easy to understand. All the constraints that help us meet these standards and keep our production flowing create back-pressure in our delivery pipeline.

We need to make deliberate decisions about where to apply strong constraints and where to remove or relax them. Apply strong constraints where they’re serving both of these goals. Don’t support them if they’re not serving one or both. Be ready to raise or lower standards as the case may require. And remember these constraints at different points in the software system are what make software quality enforceable.

We should apply strong constraints where they’ll serve this dual purpose best and consider removing or relaxing constraints that aren’t serving either purpose well. We should also be ready to shift quality bars up or down as needed. In effect, these constraints at various points in our software system are what give quality its teeth. In many cases, we can create more back-pressure and more constraints by deploying new tools or strengthening tools that are already in place. All of these things can be used to push back on most change requests. We want to build them throughout the pipeline. We don’t want to wait until the end of the pipeline when our CI system will simply tell us that we’re not allowed to deploy without fixing problems. We want to use these signals as early as we possibly can, through every possible pathway. The ultimate constraint in this system is the one we place on ourselves to stand behind the decisions and actions we’ve taken to build the system and to operate it. But like all other constraints, we need to make thoughtful trade-offs about how much we want our own judgment to restrain, to back-pressure, and to act as a final check.

Quality is in the constraints that we place around our agents. So as you’re thinking about quality for your own apps, take this problem statement and come up with your own constraint-driven plan.

This article was rated 100% human written by Pangram 4. A reminder that if you’re looking for a good starting point for setting up quality gates, Sonar offers a pretty good solution.

This idea of the software factory is a term that dates back to Bob Bemer’s paper, “The economics of program production,” given in 1968. For half a century, many have dreamed of a world in which software is a repeatable and instrumentable production process (analogous to stamping out car parts in a factory) rather than the isolated craft of individuals. Historically, this dream has generally (although not universally) fallen flat, in part because of the difficulty of stamping out ideas.

But in the last two years, things have changed dramatically enough that now it makes sense to take a fresh look at the old dream. And since some subtleties can easily be glossed over, it’s worthwhile to be somewhat precise about exactly what is really new and different, and what may be recurring traps, dressed up as new opportunities.

Dex Horthy, co-founder of HumanLayer recently gave a great talk at AI Engineer World’s Fair called “Harness Engineering is not Enough: Why Software Factories Fail.” worth checking out on this topic.

The loop is the atom. The factory is the loop at scale.

Structure is everything, and it all starts with small units. The whole stack is really three concepts layered on top of each other: the loop, the harness, and the factory.

A loop is one agent doing a single job on repeat: gather context, take an action, check the result, and go again until some condition is met. It is the smallest unit of agentic work, and everything above it is just loops stacked on loops.

The point of loop engineering is that you stop prompting the agent turn by turn and instead design the small system that prompts it for you.

A harness is the walls around a loop: the sandbox it runs in, the tools it can reach, the memory that survives between runs, and the gates that decide what “done” means. The loop is the behavior; the harness is the environment that behavior runs inside.

Hand a raw model no harness and it will happily spin forever. The harness is everything around it that makes it useful and safe to run.

A software factory is many harnessed loops running at once, fed by a queue of work and drained through a review gate into production, with humans owning the whole thing from above. It is not a bigger agent; it is an org chart made of loops.

The final paradigm shift is moving from writing code to building and running the factory that writes it. The unit of work shifts up a level, to the loop, the harness, and the flow between them, rather than the individual code diff.

The factory, drawn

The central slide Dex spent most time on was brilliant because it’s a clarifying wiring diagram that visualizes what otherwise is an obvious loop. Here’s my take on it:

Intent flows from the vision of engineering leadership, and directly from engineers, into a queue of things to be done. Signals driven by incidents and user requests drive the same queue. The harness is just the thing that picks an item from the queue and builds a change for it. Beyond the harness, we can see all the automated checks required to make changes safe enough to let into production. These automated checks run at once, effortlessly, without any conscious involvement from engineers, thanks to CI, tests, static analysis, and scanning of all kinds. The only decision point here is the review gate. After approval, changes are deployed and monitored in production, with monitoring data feeding back into the signals that kicked the loop into motion to begin with.

By and large, every box in this diagram is almost zero cost: generation, tests, scanning. They all run at scale for negligible cost. There is only one expensive box that proves stubbornly resistant to scaling, and that’s the review gate. That shiny amber box is “judgment”, and where the crux of the argument about whether we can make development faster and more frequent resides.

Why we call it “dark”

A dark factory runs with the lights physically off, because the only things on the floor are machines and machines don’t need light to see. A dark software factory is the same move: code ships that no human has read, verified only by other machines.

The image is borrowed from manufacturing. Its origins are physical rather than digital, rooted in facilities where the lights are turned off and the work is carried out by robots. FANUC in Japan has been running lights-out factories of this sort since 2001; Xiaomi, in 2024, opened a heavily automated dark factory of its own. What these have in common is a product assembled and shipped without a single human having read any of it. The “dark” comes in when that act of reading is removed from the process.

I’m not borrowing the concept for its vibe or as an insult. For all its creepy buzz, “dark” here is a simple physical claim: the original factory floor, but without light. In software, the floor is the diff. Whoever wrote the diff, whoever reviewed it, whoever shipped it, those humans are gone, and what remains is a diff verified only by the machines that built it.

This is a surprisingly easy thing to do, at least at first. It’s easy because that missing review step gets in the way of everything. Its absence makes your perception of your team’s vertical throughput seem suddenly and radically higher. It feels as if you’ve broken the sound barrier. For all its apparent ease, it’s harder than it seems to survive those dark workflows, with all their buried costs.

Harness engineering is not enough

The harness of orchestration, sandboxed prototyping, and tool calling as models interact with the world and each other will become increasingly powerful and effective. However, there’s an inherent in-model failure in trying to keep up with codebase quality over the long game and through additive changes, and I think there’s good reason to believe that models alone will ultimately lose that battle against comprehension debt.

Comprehension debt is the widening gap between how much code exists and how much any human still understands. A dark factory doesn’t pay it down; it takes it on as fast as it can, with the tests green the whole way.

This is an important distinction because models do well at some tasks. But for anything that isn’t an immediate change to a small part of a codebase, especially in a complex brownfield system, model-only automated coding faces an insurmountable obstacle. Weekend toys and side projects are alike in that a few months of development cycles is usually enough to get things in working order, or at least close enough. But an enterprise system that has been under development for a decade or more is a different beast; it has to be maintained, in a professional environment at a professional pace. Three to six months into a project, you’re already drowning in unread code. That kind of environment, and especially the constraints enforced by production code, would make even a powerful agent do poorly, all of it in contrast to the vibe-coding enjoyed by developers working on weekend toys.

Dex reports from experience that this is a major failure, so much so that it required painstaking manual debugging to pinpoint. This came from running a fully automated code factory for about four months, during which no human looked at the code that was written. Underlying the experience is a tradeoff between two conflicting metrics. One is maximizing token utilization, the number we currently treat as progress. The other, which it quietly minimizes, is the amount of the system any human participant still understands at any moment.

Where the dark factory truly shines is in its ability to burn through pristine code while the tests stay green. The ultimate reckoning, when it comes, will not be a dramatic “it all goes sideways” moment. It will be quiet and late.

The bottleneck was never generation

The fundamental constraint in a software factory isn’t how much code we can churn out: it’s how quickly we can verify it.

Back pressure is the rule that you can only hand a loop as much autonomy as you can cheaply and reliably verify, and not one inch more. Verification, not generation, is the real constraint on a factory.

Because unbounded generation capacity is in perpetual tension with the finite, non-scaling resource of human attention, the core problem is the gap between cheap generation and bounded review. Look at the funnel: as long as the neck representing verification doesn’t widen, it’s going to back up. As Dex points out, volume alone isn’t the problem: what we’re really suffering from is a surplus of bad PRs. When you’ve got high volume without trustworthy gates, manufactured defects are unavoidable. This is just back pressure again: autonomy can’t expand beyond what can be cheaply and reliably verified.

The second-order problem is why improving the model shouldn’t automatically close the gap between what it can generate and what can be verified. Training on well-architected systems is an arguably more difficult proposition than passing simple tests: remember, the cost functions measuring architectural excellence aren’t measured in seconds or even minutes, but in months and years. Tidy gradients are functionally impossible to compute, so a system expecting crisp, instant evaluation of complex design decisions isn’t going to be trained on good examples.

Turning the lights back on

A lit factory is the same pipeline with the lights left on where judgment lives. The agents still do most of the building, but a human reads what comes out before it ships, and the lights stay on wherever a wrong call is expensive.

The lit version doesn’t tack review onto the end but moves the point of human judgment upstream, to the product, the design, and the architecture before an agent starts a loop.

One great thing about that upfront hour is that it leads to fewer implementation hours. It turns a long, frustrating code review into a quick read of a two-hundred-line plan. You get to review a decision before it’s built, so later you aren’t chasing through two thousand lines of generated code to find out what the decision even was. Some decisions are expensive and long-lived enough that you’d want a person in on them early, before the cost compounds. Of course, there are still times you look at diffs, even when you’ve spent time up front.

You might be thinking that all sounds unglamorous. You’re right. The safety net is made up of perfectly ordinary architectural practices we’ve always known about and mostly ignored: good types and method signatures so that mistakes are caught by the compiler instead of in production; test seams where we can pin behavior and make change observable; laying out the code so the next reader, human or model, knows where to find the thing they care about; keeping call stacks short and legible; keeping component boundaries well defined so a change doesn’t have a huge blast radius; and dependency injection so we can swap out one piece for another. None of it is new. We’ve always said we care about good architecture. But now that we’re using automated coding agents, that architecture is finally doing a second job as a cheap and hard-to-fake safety net against the mistakes the agent will make.

That safety net has to live outside the model, because the model won’t supply it. The coding agents that feel most capable, Claude Code and Codex among them, are reinforcement-trained against their own harness and tools: fluent with all the tools and idioms of the trade, but not with things like long-term maintainability. The deliberate architecture we’ve always talked about is the tool that catches that debt, and the investment we make in it is us buying back our autonomy. Put that together with safe infrastructure, and there are some tight, low-risk loops you can run unattended. Horthy described one in a recent post: a nightly GitHub Actions cron that fixes exactly one anti-pattern, a lint violation or a needlessly optional prop, commits, and opens one small pull request, all on its own, so the team wakes up to a slightly better codebase and a diff short enough to read. But for loops with high enough stakes, you don’t want to risk waking up to a broken auth system, billing engine, or public API contract. Keep the lights on there, and trust that a person with judgment and a real working knowledge of the system will catch the mistake.

What earns a loop the dark

This rule applies whether you call it back pressure, verification, or the light switch.

A loop can earn itself fully automated status only if the check is cheap, runs at high frequency, and relies on something that can’t be easily faked out. Green-or-red oracles, type gates, property tests, and a review agent coupled with a real rubric all fit. You also need the oracle to answer immediately and not drift over time. When done can be proven not just by you but by a machine, you’ve reached automation.

Short loops are easier to verify than long ones. Dex’s rule of thumb: an agent holds up for three to ten steps, then starts losing the thread past twenty. The reason is context accumulation, the more the agent drags along, the more likely it is to wander off. When a loop is short, verifying it is cheap. Sprawling loops hide mistakes in the corners, which is another way of saying they never earned lights-out status.

Keeping the lights on is the opposite case. A loop needs to be reviewed if a wrong answer is expensive and only a person can catch it. Subtle production bugs that can’t be caught by tests, large blast radii, and a decision that’s going to shape the work of a year or more all qualify. In those cases, your attention is the actual product, the costly, essential one.

The danger is forgetting to flip each switch and just setting all of them to the same mode. All dark, and you’re stuck tearing everything down four months later. All lit, and no one can get reviews done in time and you’re stuck in a gigantic bottleneck. The hard, skilled job is deciding where to put each switch.

Loops, graphs or state machines?

When you hand an agent a task, you’re probably going to build a graph around it, whether you call that graph a finite state machine or a set of conditionally-linked service calls. It’s a framing where the software isn’t just following some abstract rules but a structured workflow: every node is an explicit step, and every edge between nodes is an explicit condition.

That sounds like a lot of structure, but most of it is already there in any software, since any code can be expressed as a control-flow graph. So the only real novelty is that an agent insisting on autonomy is really just walking around a particular graph, and its freedom is constrained to the inside of a node. And here’s the part people forget, which Dex wrote down a year ago: software was always going to have that structure. There’s a reason we used to draw programs as flow charts. The genuinely new move was trying to throw the diagram away, leaning on a loop where the model picks the path tool call by tool call, until it declares itself done. That felt like liberation, right up until it met a ten-year-old codebase, and the discipline everyone is now rediscovering, owning your control flow, is really just walking the graph back around the loop. So the question of whether we should shift from loops back to graphs is almost an admission that we needed the flowchart all along.

Here’s what it looks like in practice. Take a bug to fix. As a pure loop, you sit down and think: figure out what’s wrong, change some code, run the tests, see what happens, and if that round doesn’t kill the run, loop back and start again. The whole journey is decided as you go, which problem you chase, the exact code you change, which tests you run and in what order, whether you run tests at all, and whether you try again or declare victory. As a graph, the first thing you do is map out what should happen. Reproduce the bug or go ask for more information, find the cause, try a fix, run the tests, and let a failing run route back to the fix while a passing one goes on to review, where only an approval reaches done. The agent is still clever inside each box; it just can’t wander off the paths you sanctioned. Santi laid this out with a diagram that makes the difference obvious.

The real appeal of that graph, of course, is that it’s back pressure drawn as a diagram. You give up some of the agent’s freedom and get mandatory checks and legible failure points in return, so when a run dies you can point at the node that killed it. It’s the same instinct behind Dex’s blunt line that most so-called agents aren’t very agentic at all, “mostly deterministic code, with LLM steps sprinkled in at just the right points.” And this isn’t just an artifact of how people happen to be building things right now: you can see the pattern in LangGraph and LlamaIndex Workflows, in Jerry Liu’s hybrid workflow-graph-over-agents with an outer loop that grows parts of the graph as it runs, and in David Khourshid’s reminder that this is really just state machines and the actor model turning up in new clothes.

One clarification, because the term is badly overloaded: when I keep calling this a graph, I don’t mean a knowledge graph. I mean a predefined directed graph of how the work should flow, conditional edges and all, giving the loop a shape you can actually trust.

Where the human actually goes

Notice that the person never left the factory. They moved.

I think engineers need to increasingly own the outer loop. The agents can investigate a bug, write up the diagnosis, implement a fix, run the tests, and write up a report. That’s the execution of the inner loop, and they can do it as efficiently as anyone. But that was never the job. The bits you own are what I’d call the outer loop: decide whether it’s the right way to address the problem, verify that the diagnosis and implementation are sound, approve the change, and carry the consequences of being wrong. The boundary between the two loops is evidence, the diffs, the tests, the logs, and a brief explanation that connects them. Types, seams, and rubrics make it possible to oversee all this without doing a lot of work for every change.

I think it’s useful to put it this way: you’re not down on the line writing changes any more; you’re up at the end of the production line designing it and guarding the gate. There’s a lot you can do to make the model better and the harness more capable, but I’ve observed that identifying problems that are expensive in the long term is not typically something you can automate away. The core thing that’s still the job is to exercise human judgment better than any flow of paper and computing power.

Robots are fine operating in the dark, but humans need to see what they’re doing. If everything on the factory floor is dark, and you can’t see anything, and you can’t even find the light switch, that’s where the danger is.

Pangram scored this article as 100% human written.

Agents have leverage, and leverage creates obligations. Someone must be able to explain exactly what changed, why it was safe, and what will happen if they’re wrong. Otherwise, their actions can’t be justified. Which makes it unlikely their organization will ask for them in the first place.

And so I want to talk about three terms. The first, Quality, refers to all the checks we install before we let the system loose. Those checks produce evidence, and from that evidence we derive a Verdict.

The second, Verdict, refers to the final decision we make before work enters our dependent system: I’m the line-producer of this content. I run the team whose work is shipped under my name. The model may write the line, but the Verdict is mine. The work of my team will not enter our dependent systems without my decision. A Verdict is the production decision: should we ship, block, redirect, narrow the response, add a guardrail, or reject outright?

The third, Answerability, refers to the guarantee that if someone asks, I can explain why.

To say this another way: our agent (which I define as a model plus a harness of files, tools, memory, skills, sandboxes, permissions, observability, and recovery) is what runs our loop (which I define as investigation, implementation, verification, and repeat). And it’s what creates our software factory.

The model is just the engine. The harness - tools, memory, permissions, sandboxes, tests - is the car you build around it so it can do real work safely.

Wrap that harness in a repeatable cycle: investigate, implement, verify, repeat. The loop is how one good run becomes a process you can trust to run again. Wrap that harness in a repeatable cycle - investigate, implement, verify, repeat - where an independent check, not the model’s own say-so, decides when the work is done.

Now run many loops at once. A factory is loops at scale - the agents ship the work inside, while humans own the decisions at the boundary.

And at the heart of that factory is a careful boundary between what’s inside the system and what’s outside it. Inside the system: we collect inputs (from the product team’s intent, or knowledge of previously shipped work, or of recent incidents, or of specific feedback from users). The agent loop investigates the task, implements a plan, and verifies the result. Then, evidence crosses that boundary. A human, who owns the dependent system, sees the evidence and decides whether to proceed.

And that, friends, is the shift we’re trying to make. Before, our agents were doing the inner loop of the execution loop. Now they run the inner execution loop. Engineers own the outer loop.

Inside the system, there’s really just one kind of thing our agents are doing: capability. The capability to investigate tasks, implement plans, test their results, and report back. That’s the capability of a model. And as we’ve said, that future is already here.

Outside the system, there’s a single kind of thing: agency. The agency to decide, verify, approve, and own.

We’re still talking about code, you see. It just needs to live in a place and be performed by people who know what they’re doing.

The potential for AI code is no longer marginal. In a Sonar 2026 survey, we asked teams about the share of their commits that were AI-assisted. It was small but non-trivial. And several of the respondents said they expect the share of AI-assisted commits to grow substantially.

Sonar’s 2026 State of Code report found that 42% of committed code was AI-generated or significantly AI-assisted, with expectations for that share to keep growing rather than plateauing.

Creation, in other words, is getting cheaper. Scarcer resources are review, validation, understanding, and maintenance.

We moved the speed of generation faster than we moved the speed of control. And so we have a trust-verification gap. A lot of people we talk to still express some degree of distrust in AI code. Yet fewer of them seem to consistently build that distrust into their verification processes.

And that’s a dangerous place to be in. We’re going to need cheaper, clearer ways to verify the trustworthiness of AI code.

If you look at the GitLab June 2026 report, you’ll see that governance questions have shifted.

GitLab’s June 2026 AI accountability research shows that review and validation are the current bottlenecks when using AI and, more worryingly, that governance usually happens after code creation, after we’ve accepted the risk and lost control over ownership. Today, it’s not just about control. It’s about what constraints we set on the system. It’s about how we’ll check the work with evidence, and how we’ll hold teams accountable. It’s about who will own what part of the AI lifecycle.

So the final distinction in this series is between process and quality. Quality is the concept of back pressure. We mean it literally. We don’t want to grant our agents as much autonomy as they can possibly exercise. We want to grant them just enough autonomy that we have enough back pressure to stop them, regulate them, check their work, and ensure our humanity.

Ordinary engineering holds up a lot of signals that indicate that the work being done is doing the right thing. Type checks, tests, hooks, sandbox limits, audit logs, monitors. Our engineering systems are full of these kinds of signals, and they’re designed to provide enough back pressure to keep the system honest.

And so as long as our agents are emitting these same signals, we can trust our ordinary engineering to provide appropriate back pressure.

Trusting our systems doesn’t mean we don’t want a human in the loop. It just means that the human doesn’t need to be in the inner loop. We want them in the constraints loop (what inputs, architectures, instructions, or invariants should we set?), the sampling loop (how much output should we sample and review?), the audit loop (what evidence should we keep and how do we make sure our audit log is effective?), and the ownership loop (what part of the production boundary should we own).

But the human doesn’t need to be in the inner loop.

The agent can ship more than you can review.

And the scarce resource is your own core human judgment, informed by quality signals like logs or tests.

The AI June 2026 report shows that, in the experimental setting, agentic delegation along hour-scale time horizons is essentially here. The work by OpenAI this year on agents and the future of work was a great source for these ideas. So we need to start thinking about how to establish this ownership boundary, as our systems start shipping more than we can review.

And that’s where the answerability comes in.

Because with long-horizon agents, the decisions made over hour-scale time horizons are just that - decisions. And not all the decisions are going to be recorded. You can’t trace them all back to input tokens. If all you’re doing is trusting that the output you get is the correct choice for the problem at hand, the hundreds or even thousands of human hours of work you’re going to need to reconstruct the chain of decisions that lead to it become impossible. And so, again, answerability becomes something that must be at the core of our system design.

Three hidden costs

And there are three hidden costs:

Cognitive surrender ~ blindly accepting what AI gives you. When you delegate work to an agent, the work itself may appear to be the work of the agent. But it’s actually your work. It’s your reputation. It’s your responsibility. And it’s your software that suffered the defects in the output. And it’s your software that needs to be changed to reflect that output. So the agent’s output becomes your answer. And with it comes all the accountability. The Wharton study that put this together is reassuring when the AI is right. But when it’s wrong, the news isn’t great. When the AI was wrong, nearly three-quarters of people accepted it anyway, and felt more confident than they would have without the AI.

Cognitive debt ~ erosion of your understanding and memory of how to solve problems. When you delegate work to an agent, you’re offloading all the thought work to the agent. And while thinking it all out yourself takes time and energy, thinking it out on a massive codebase takes resources that aren’t available when you’re trying to run up the learning curve. So the output you get is often unattainable by you. And the longer the time horizon of the agentic planning, the bigger the gap between the code the agent produces and your understanding of it becomes. The gap compounds. The debt accumulates. And the cost of climbing the learning curve grows almost exponentially. There’s a randomized controlled trial from Anthropic

looking at whether engineers who lean on AI to write code understand it as well as engineers who write it themselves. The conclusion was gloomy: on a comprehension quiz, the engineers who worked through AI scored seventeen percentage points lower than those who didn’t, 50 percent versus 67 percent.

And then there’s the orchestration tax ~ its easy to spin up lots of agents now, but your cognitive bandwidth doesn’t parallelize in the same way. Steering your agent away from the worst behaviors, sorting the work the agent produces to identify the ones that need your attention, directing it to focus on the work you care about first, verifying your most important constraints and your most dangerous assumptions before you let it run…

All of that takes work, and it can’t be automated.

There’s no substitute for human judgment.

Brownfield systems are especially dangerous here, because the system behavior you have to audit doesn’t live in the code. It lives in the scars.

Fixes? Make attention the priority in your architectural decisions. Use worktrees, scopes, and evidence to reduce the coupling between your initial plan and the work that emerges from it. Time-box the effort to resolve unactionable steps. And make change in your software strictly an opt-in permission.

Alpha, decay, and taste: these are the three core patterns that shape careers and performances across domains.

Alpha is the lead part taken up by the highest achiever in the competition, when you’re playing your highest-value game move. Decays are established patterns that everyone learns through repetition and watching others (plateaus, if you like). Taste is the earliest we can sense the lead in an alpha or the change in a decay. It’s our judgment of what’s coming before we have any evidence that anything is happening.

Paul Graham’s point is that when anyone can make anything, choosing what to make matters more, and Mitchell Hashimoto’s definition is the operational one: making high-quality qualitative judgments where no objective metric exists yet. From now on, taste drives everything: alpha shifts are taste changes. And decays fade out because we start to taste something different.

Next step? Operationalize your taste. How? Give it a name that reflects what you’re trying to move from limbic to conscious. Practice it in critique and examples. Make its rationale explicit.

And keep making the move that delivers the most durable competitive advantage in your industry. What’s that? Keep moving the edge up from just doing the task to teaching it, systematizing it, deciding when it should be done, and owning the result.

Everyone is a developer, but not everyone is an engineer. Engineering is what a developer turns into when they embrace a work discipline that is more strict: thorough and logically sound reasoning, consideration of constraints and tradeoffs, recognition of risk and exposure, and practical accountability.

In the future, people will leave the administrative work of engineering and embrace new roles that emerge as engineering becomes more demanding. Roles that are un-bundled from the spirit of craft but make clear what each person does. There will be those who prototype. Those who build. Those who sweep. Those who grow. Those who maintain.

The humans hold the edge of the system in the other direction, too. Increasing the alpha in the other direction: choosing what is worth doing, defining the constraints within which it should be done, deciding if the evidence is sufficient to proceed, and caring for the result. Whether it’s a single team or a hundred teams, this is the edge that only humans can hold.

Accountability will scale the factory. Like attention and taste, accountability is also one of the three dualities that makes everything work. Without accountability, there are no rules. No wrangling with questioners. No trade-offs. No risks. No safety nets. If nobody owns the consequence of a decision, then high agency can only bring chaos.

The half-life of an edge is one release, but the half-life of a signature is a career. A signature is your name on the work, such that you feel you can stand behind what was shipped. Skills get you leverage; accountability turns leverage into trust.

Only people can choose. Only people inherit consequence. Agents can be asked to choose, route, merge, and escalate safely inside a policy, but they cannot inherit the consequences.

Every codebase should perhaps come with some kind of accountability contract that explicitly states the checklist that was understood when the change was accepted, the evidence that went into the decision, who was accountable for the change, and the system status after the change was blocked. Just like:

• Your attention and taste

• Your evidence, verdict, and ownership

• Your alpha, decay, and taste

High agency

In a typical agentic workflow, high agency is the art of knowing when to delegate, when to inspect, when to stop, and when to own the result of a process. The ladder of agency runs from low to high: flag a potential problem, investigate it, execute against it, diagnose it, propose solutions, recommend fixes, and resolve the issue. A high rung on the agency ladder is discernment: found it, it’s not worth fixing, moving on.

The twelve pillars that hold up the software factory

Brownfield is the frontier for factories that hope to scale. All those clever little innovations may not feel like much yet, but the production environment is a lot. When building an entirely new system, it’s much easier to plan and implement sufficient back pressure mechanisms because you have full control. When you’re adding intelligent agents to a legacy system, however, it’s another matter entirely.

Legacy systems include the entirety of production behavior, future expectations from customers, migration histories, release and budget cycle durations, unspoken assumptions, edge cases, data weirdness, runbook procedurals, and all the scars that accumulated without the will to care for the system.

To be a steward of brownfield requires a form of durable engineering. Work has to be done to turn implicit knowledge into explicit constraints, keep it coherent across teams and through generations, formalize that knowledge into test procedures and functional specifications, and tie that knowledge to objective evidence. All while ratcheting failure into more learning. Because if the system doesn’t get the care it has always received, everything will come crashing down.

New Work is Real Work

The work will get more interesting as you scale. Because when everything else is built, people will want to build new things. They’ll want to employ the alpha and taste they have developed through their craft to design new loops that can be grafted onto the software factory. Or they’ll want to build greenfield systems that employ all the knowledge of the software factory to one elegant, well meaning, principled effort. They’ll want to design and implement new forms of evidence that will rise to the level of verification for the new systems. They’ll want to take care of brownfield systems that are now so complex they need dedicated attention. They’ll want to design and manage new back-pressure mechanisms. They’ll want to design new agents. And they’ll want to build agency.

And, as they do, they’ll come to see that all this is real work. That’s a good thing.

Automation creates bottlenecks. Bottlenecks in production that are worth owning. Because automation gives us control over industrial scale. But there’s also new bottlenecks that arise from industrial scale. The bottleneck moves from “can we build this?” to “should this exist, can we answer for it?”

What I’m suggesting is a practical operating model for scaling agentic engineering. There’s inner and outer loops. The inner loop is where the work is done. Loops are designed to be as independent as possible. Put all quality assurances and verification inside the loop. Once you’ve designed and validated the loop itself, the only thing you have left to do is to grant autonomy by putting in place a back-pressure mechanism that acts to control the rate at which the loop is run and its scope of operation. And put humans in their rightful place, on the right decisions. Don’t treat understanding as a hand-off or a release gate, but rather as a point of decision where humans are primed to provide their insight. And then for every artifact that exists and is fed back into production and into new teams and engineers, leave behind better artifacts.

Build the factory; keep the lights on; make work legible, verifiable, owned.

An agent can write it. But before it reaches users, someone must explain why it should exist, why it’s safe enough to be part of production, and what they will do when it is wrong.

That is agentic engineering at the outer loop - that is the work now.

Pangram has scored this article as 100% human written. It was cross posted to Twitter/X.

From the engineer standpoint, you’ll use low autonomy to limit risk and increase reversibility, but use higher autonomy for explicit activities, and fleets of parallel agents safely refactoring massive codebases. The core question about an action is always: what level does this task deserve, and what verification makes that level defensible?

The edge of the frontier is the manager agent that wakes on its trigger, delegating to its helpers while continuously verifying their output, and returning with only the decisions that must be made by a human. Folks using this kind of setup may indeed already be running hundreds or thousands of agents, largely on evergreen codebases. Like most all thinking about autonomy, how you perceive the scale is still different for everyone.

The scale most often mentioned is from Steve Yegge’s single-axis ladder mentioned in “Welcome to Gas Town” and in The Pragmatic Engineer. It’s a good reference if you want a number that tells you how AI-native you are: the ladder gives you a single number to measure if you know your trust in a single agent. Here’s one version of it:

In early 2026, even while work began to shift from delegation to orchestration, this was a fairly good proxy for measuring risk. Today, however, many skill sets may have increased significance and leverage when you can run many agents at once. A single rung cannot help you place multi-agent skill.

Instead, almost every autonomy debate I’ve seen conflates two questions that should be separated: how far away from yourself are we letting this single agent go, and what is our skill at coordinating many agents?

To capture these two dimensions separately, we’ll use two axes: agency and orchestration.

On the agency axis, low includes suggesting candidate actions and waiting for a decision.

Mid means that the agent is working on a particular task, but scopes what it does, and constantly reports back what it does along with evidence, so you can keep steering it.

At the high agency end, the agent is working towards a goal, experimenting, learning, testing, finding ways to solve a problem, getting blocked, asking questions, trying different approaches, and returns all of this work in evidence.

On the orchestration axis, low means one agent, one thread. At mid, you’ve got several agents, each working in its own worktree, possibly working towards different goals, but isolated. At the high end, you’ve got an orchestrator that can take a backlog, issue tracker, schedule, or other queue, and turn it into continuous work, and you only need to step in when things fail: “management by exception.” Products and features incorporating these ideas include:

• Claude Code’s /plan, /goal, /loop, /background, /batch, /code-review, /security-review modes, subagents, hooks, checkpointing, agent delegation and management practices, background sessions, agent-team patterns, /schedule arguments

• Codex’s local/cloud threads, Goal mode, worktrees, Automations, subagents, review panes, GitHub code review, hooks, sandboxing, Auto-review, and rerun

These capabilities don’t fit onto a single ladder.

The climb: three eras and a single stack

If you read the ladder bottom-up, you’re imagining climbing both agency and orchestration at the same time. In effect, the six levels represent three separate eras that we all pass through:

First, you’re in the driver’s seat, and an agent mostly just helps, waiting for you to steer it.

Second, the agent takes charge of a bounded task or goal, but you’re still around to steer it and verify what it does.

And third, in the era of orchestration, the system is capable of running the show, dispatching work across many agents, and you mostly need to step in when things go wrong: “management by exception.”

This makes things simpler, because the vertical position on the ladder neatly captures the two axes (orchestration only kicks in near the top), leaving it as a single steady climb through the rungs. And yet, the climb is still part of a shift that we’re all going through.

A good day doing engineering includes touching several rungs, sometimes more: it’s normal to switch between the eras a few times in the course of a task.

The six levels in detail

Level 0: Assist

The agent makes suggestions that are mostly good and often perfect, but you will always decide whether they’re good enough to act on. Think autocomplete, inline edit suggestions, or hanging around in a chat session discussing a change that nobody has taken ownership of yet. Use for costly errors, tiny changes, or when you’re forming your own judgment. Verification mostly takes place locally.

Level 1: Supervised action

The agent edits or runs commands on your behalf, asking you before executing anything consequential. This is the default posture for most people. It can be done in local sandbox with approvals before applying changes - where each approval is an independent verification that the change is okay to apply - or in an interactive session. Failure mode is approval fatigue; all approvals feel the same regardless of what they’re approving. You might solve this by squinting at the diff, following some heuristics, checking in with another person before approving, or just agreeing to let the agent be responsible. Codex Auto-review solves this problem by delegating the final approval of boundary conditions to a separate reviewer agent.

Level 2: Scoped task delegation

Hand off a bounded task to the agent. That task will have a clear goal, constraints, and a working definition of what done looks like. You’ll stay nearby, able to interrupt, but mostly not involved. This is the center of gravity in the software engineering world. Verification is shifting away from you (you may need to rest and sleep) towards evidence that the agent can produce: passing automated tests, proper types, lint suggestions, screenshots, repro steps, provenance by example, etc.

Level 3: Goal-driven autonomy

The agent does whatever it takes to achieve a goal, stopping only when some condition is met. In prompt mode, this means the prompt itself becomes the goal (e.g., “Can you reduce this page’s time-to-interactive below 1 second?”). In Codex, this is Goal mode: the agent cycles through plan->act->test->review steps until it stops meeting success criteria. In Claude Code, it’s the /goal, /loop, and /schedule commands. For this level to be useful, the stopping condition must be measurable in a way that can be automated.

Don’t ask your agent to help with vague, wooly goals like improving user experience in general” or “make the codebase more testable.” Pick something specific, measurable, and automated: find bugs in production that elude static analysis, reduce load time, ensure that we have a strict TypeScript build with no explicit anys, triage all dependencies to keep just those that we understand and which pass our tests, etc. And, finally, to find bugs in production, the agent will need to be in a production-like environment.

Level 4: Parallel delegation

Work across many agents in parallel. Each agent works on an isolated slice of the task. The biggest bottleneck at this level is decomposition: defining the right slices to delegate. Supports include: subagents, background sessions, /batch, worktrees, agent teams, etc. Failure mode is false parallelism: running many agents against overlapping slices at once, so instead of more work you get merge conflicts and duplicated decisions. To do this well, agents need to be isolated from one another, each owning their own files and status. Each needs to have its own review queue, as well. And finally, each agent incurs a cost - in terms of tokens consumed - proportional to the number of agents running at the same time. On the human side, orchestration tax makes the marginal cost of adding an agent go up after a few.

Level 5: Managed-by-exception orchestration

Define what success looks like, and which policies should apply. A manager agent will wake up based on triggers (e.g. new issue, new task, clock), dispatch worker agents, monitor their progress, verify output, retry on failure, escalate to more competent agents or humans when conditions are met, aggregate results, and ultimately return work products (e.g. PRs) and evidence to external systems. Think factory: the issue tracker or backlog is the input, and the product of the factory is the output (i.e. many fixed issues, bugs). Agents work in an appropriately isolated environment with lots of walls (and if needed, escape hatches), and only an operating system - defined by the manager agent - defines what the factory is expected to do.

The design of this operating system is left to the human; OpenAI has proposed a spec for Symphony which has a Linear board at the center: each issue gets its own agent workspace, and the agent continuously ensures that it is making progress towards its goal as defined in a spec file in its own workspace. Human review can be done at the altitude where evidence is generated, but the frontier (i.e. what is most powerful in the orchestration world) is to build continuous agent factories with hundreds or even thousands of agents. At this point in the climb, it becomes increasingly important to have independent verification: separate implementers and reviewers, separate test runners and QA, separate security checks, separate process gates for acceptance.

Risk and reversibility set the ceiling.

I remember reading an earlier Anthropic study on some of the hardest tasks with Claude Code where it asked for clarification more than twice as often as users interrupted. Experienced users (~750 sessions vs under 50) were more likely to auto-approve and interrupt keeping an eye on the progress.

They also did a lot of broader analysis of how people use Claude Code. They looked at ~400K sessions from ~235K people between October 2025 and April 2026. From each session they could figure out the decisions someone makes like how many actions they ask for in each prompt, which of these they choose to auto-approve, how often they interrupt etc. People make ~70% of the planning decisions, but Claude does ~80% of the execution. High autonomy is not about leaving people out of the loop, but moving from having them do every step to having them decide which direction to go next.

If we want to determine whether a large AI system is operating with high autonomy, the three questions we should be asking are:

• How quickly will we know we’re wrong about what it’s doing?

• How cleanly can we undo what it’s doing?

• What would prove we’re right about what it’s doing?

If the answer to all three is: not quickly, at great difficulty, and trusting the summary, it’s not high autonomy.

Every run of an agent should be preceded by a contract that defines what it’s trying to do.

The goal: what we’re trying to achieve (not an activity, not the technique, but an outcome).

The scope: what domain we’re operating in, and what techniques are allowed.

Non-goals: what isn’t part of the objective.

Tools and permissions: how the agent can interoperate with the world. Stopping condition: when to stop; ideally, a measurable variable.

Evidence: specific tests, screenshots, logs, database records or other indicators that can be used to confirm something has been done (independent of the agent).

Escalation: who gets involved in what circumstances (including who runs the agent).

And budget: a limit on how much time, effort and tokens are to be devoted to the task (tokens are the budget of large AI models - you can also include a limit on the number of times it can attempt the task and a limit on the degree of parallelism).

Metrics make autonomy just a little more reliable

Deciding on metric after-the-fact is probably not enough. Metrics can be put in place in advance, in a concise doc. And that makes autonomy feel more reliable and makes the leap of faith just a little easier to take.

While there are many ways to measure success, considering tracking some version of these metrics for each level of autonomy:

• Mean time between interventions

• Longest successful unattended run with accepted work

• Share of actions run in the sandbox vs escalated

• Percentage of actions auto-approved vs rejected

• Mean number of agent actions per human instruction

• Clarification request rate Interrupt request rate

• Review time per accepted change

• Rework rate on each level of confidence

• Defect escape rate on each level of confidence

• Token cost per accepted change

Such metrics can tell a story: a single agent kept busy by human handoffs is Level 4 with a dashboard. A conservative agent, unwilling to proceed without automated intake, retries, and decent evidence, is Level 5 with a real gate.

Think about readiness

Classify work by risk and by how easily it can be undone. Apply autonomy conservatively, rising only as evidence supporting the higher level accumulates. A payment engine refactor protected by strong tests and reviewer agents with a clean rollback path can support much higher autonomy than a documentation automation task lacking any canonical truth. The autonomy level should follow the verification process, not the task name.

Four anti-patterns

Every system can easily fall prey to these four autonomy anti-patterns unless vigilantly avoided.

Autonomy as status - an agent’s autonomy rating becomes a meaningless badge of status. Higher autonomy is treated as proof of capability, not of safety, and agents are run hotter than verification supports. Fix: Praise and reward those who settle on the correct level of autonomy and relentlessly avoid overstepping.

Permission laundering - the tyranny of approval fatigue leads us to grant AI agents and tools wildly broader access than necessary. Fix: Better boundaries are always a fix, such as sandbox profiles, scoped writable roots, allowlisted commands, hooks, and Auto-review.

Summary substitution - the agent’s work summary substitutes for review, assuming the summary is sufficient. Fix: Bundle the same evidence packet as with fully manual reviews (a diff, tests, logs, screenshots, reviewer findings, risks, gaps, etc.) while avoiding cognitive surrender.

Fleet cosplay - dozens of agents run in parallel, but a human persists in orchestrating every dependency manually. Fix: Shared state, ownership rules, and better dependency tracking gradually reduce the need to coordinate manually. Smaller WIP limits force a focus on encoding and documenting the coordinated steps until orchestration becomes automated.

A calibration exercise

Review the last ten tasks you undertook with agent assistance. For each task, record the autonomy level exercised, the risk involved, how easily the work could be undone, the evidence produced to meet verification requirements, the review time, if any rework was needed, and whether the autonomy level chosen would still be a fit next time.

How to climb safely

Move up one axis at a time. Start with a single supervised agent to do a single scoped task that produces defensible evidence of success (an autonomy level 1, if tidy enough). Then gradually expand in the three orthogonal directions. Parallelize read-heavy exploration tasks (autonomy level 4). Add write agents acting on separate worktrees with constrained file ownership rules (autonomy level 4). Add recurring automations, then agent-led orchestration based on issues, voice, etc. Every step up the lever requires a new set of safety mechanisms (such as new failure modes).

Name them: Longer single-agent runs can lead to drift, context rot, dropped communication, or strayed objectives. Background work can lead to stale assumptions and weak handoffs. Too much parallel work can lead to merge conflicts or duplicated decisions. Too much recurring work can lead to silent token spend or stale prompts. Managed by exception can lead to long review queues and alert fatigue. Fix not trusting harder; instead, narrow scope, ensure better evidence, enable cheaper rollback paths, harden gates, and define clearer ownership rules.

Use the autonomy level:

• Level 0 is best for delicate work and when judgment is still being formed.

• Level 1 is best for most exploration, if the work is done close to the boundaries of what is well-understood.

• Level 2 is best for most bounded tasks, knowing there may be unknown depends and unforeseen gotchas.

• Level 3 is best where the success conditions can be stated with sufficient clarity.

• Level 4 is best when the work can be cleanly split across these success conditions.

• Level 5 is best once the coordination and communication needed across the various success conditions is fully encoded.

Verification will always be the bottleneck.

Despite current bravado and current tooling, the mature posture of an engineering team working with AI agents is calibrated autonomy.

In the near future, we’ll want to design loops that know when to work, when to verify, and when to ask - but the skill of the engineer will still lie in choosing the right level of autonomy and in building patterns and defensible evidence that guard against its darker corners.

I am more optimistic about agentic engineering than I have ever been. The agents are genuinely good, they get better every month, and on an ordinary week I now ship things I would not have attempted in the same time a year ago. This write-up is a map of where the interesting work went, because it did move, and most teams have not fully caught up to where.

Code review used to work because of a happy accident of relative speed. A senior engineer could read code faster than a junior could write it, so review kept pace without anyone designing it to, and the team absorbed how the system fit together as a side effect of reading each other’s diffs. A lot of that was not deliberate. It fell out of a single fact: writing code was the slow, expensive part, and reading it was cheap and fast.

That fact no longer holds. An agent will produce a thousand lines of often solid, well-formatted code in less time than it takes me to read this paragraph, while a human’s reading speed has not changed since roughly the day we started staring at screens for a living. So the constraint moved downstream, to the one step that did not get faster: a person being confident the change is right. I do not think that is a loss. It is the most leveraged place in software to be good right now, and it is where I have put most of my attention this year.

There is a happy twist here that shapes the rest of this piece. The same tools generating all that extra code are also the best thing I have for keeping up with it. On my own projects, including the popular open-source ones, I now point Claude Code or Codex at a batch of incoming PRs and have them triage the queue for me, and that has genuinely changed how I spend my time. So this is not an anti-AI argument, and I will come back to exactly how I use it.

It is also not a data dump, and not another round of whether letting a model write your code is wonderful or the end of the craft, because that framing is useless. The only answer that survives contact with a real codebase is that it depends entirely on who you are. A developer vibe-coding a side project a dozen people will ever run, and a team keeping a ten-year-old enterprise system alive for another quarter, share almost no constraints worth naming, and most of the advice in circulation is really one of those two people telling the other how to live.

What the 2026 data actually shows

The productivity gains from AI are real, but raw output overstates them: about four times the code for a tenth more delivered value. The gap between those numbers is review work, which is exactly why review is where the leverage now sits.

For a couple of years this was anecdote and argument. It is now measured at scale, by organizations with no shared agenda and in several cases competing commercial interests, and the measurements keep pointing the same way: AI pushes output sharply up, and pushes both quality and reviewability down.

Faros AI instrumented 22,000 developers across 4,000 teams and tracked what happened as teams moved from low to high AI adoption. This is March 2026 data, about as current as anything here. The upside is real and worth stating plainly: developers merge considerably more PRs and complete more work, and throughput per engineer climbs. Then the rest of the report:

• code churn up 861%

• the incidents-to-PR ratio up 242.7%

• the per-developer defect rate up from 9% to 54%

• median review duration up 441.5%, with time-to-first-review and average review time both roughly doubling

• PRs merged with zero review up 31.3%

The last figure is the one I find hardest to dismiss, because nobody chose it. There was no decision to stop reviewing. Reviewers simply could not keep pace with the volume, so code began merging unread, and that became normal. The detail I keep returning to is that teams with mature, disciplined engineering practices were hit just as hard as everyone else. Good process did not protect them, because the volume arrived faster than any process was designed to absorb.

One caveat to hold throughout: CodeRabbit and Faros both sell into this market, so their framing is not disinterested. That does not make the numbers wrong, the effect sizes are large and consistent across unrelated sources, but vendor research deserves to be read with that in mind.

CodeRabbit studied 470 open source PRs in December 2025, 320 AI-coauthored and 150 human-only, and found the AI changes carried roughly 1.7x more issues: logic and correctness problems up about 75%, security issues 1.5 to 2x more common, readability problems more than tripling. Their AI director David Loker described these as “predictable, measurable weaknesses that organizations must actively mitigate”. Predictable is the operative word. These are known, locatable weaknesses, which is good news: it means a review process, human or automated, can be aimed straight at them.

GitClear has interesting data here too. In their productivity data through 2025, daily AI users produce around 4x the raw output of non-users, but measured against their own output a year earlier, the real productivity gain is only about 12%. You are generating roughly four times the code for something like a tenth more delivered value, and a human still has to review all four times of it. To GitClear’s credit, Bill Harding is explicit that some of even that 12% is selection bias, because stronger developers concentrated in the AI cohort. The gap between 4x the code and a tenth more value is the review problem stated in one line.

GitHub reports that Copilot review has now run over 60 million reviews, a 10x increase in under a year, and more than one in five reviews on the platform involves an agent. This is no longer a niche practice. It is how code gets made.

Four datasets, four methods, one conclusion. We poured machine-speed output into a system built for human-speed work. The bottleneck did not disappear; it moved to verification, and review is where that bill comes due.

Everyone is solving a different problem

How much review a change needs depends almost entirely on its blast radius, and most advice you read was written by someone operating at a very different one.

Almost all the alarming data above comes from enterprise telemetry and from open source maintainers being overwhelmed. It is entirely real if that is your situation. If you are one person shipping something a handful of people will ever run, much of it simply does not apply to you, and you should not be made to feel otherwise.

Three variables determine where you sit:

• blast radius: what happens when it breaks. Nothing, or angry users and money and PII on the line.

• how long the code lives: a throwaway prototype you might rewrite next week, or a codebase you will maintain for years.

• how many people need to understand it: just you holding the whole thing in your head, or a team that has to share ownership over time.

Run the same diff through those three and “good review” means genuinely different things.

If you are working solo on a greenfield project with no users, review’s second job, distributing knowledge across a team, does not exist for you. You are the team.

The reasonable move is to lean hard on tests and automation, review the parts that genuinely matter, and accept a lighter touch on the rest. Duplication and churn cost far less when the code may not exist in a month and nobody is paged at 3am when it breaks. The catch, and people learn this one painfully, is that it only works if the tests are real. Skipping review without a safety net does not remove the work, it defers it at a higher price, and standards slip when no one is there to push back. No users is permission to defer review. It is not permission to skip verification.

Then the project gets users. This is the dangerous middle, and the crossing is rarely noticed at the time. Review’s bug-catching role suddenly matters, because bugs now hurt people, and its knowledge-sharing role switches on, because it is no longer only you. Teams keep their solo-era habits a few months too long, and then there is a postmortem and the Faros numbers stop being a chart and become their own dashboard.

At the far end is the large organization with an old codebase and many users. Here every alarming figure lands at full strength. A change nobody understood is comprehension debt that becomes someone’s on-call incident. Review is doing several jobs at once, and the volume of agent output quietly breaks all of them. The Faros finding about mature teams is aimed squarely here.

So the point is not “enterprises should be cautious and solo developers can relax”. It is that the purpose of review changes with your position, so the rules have to change with it. Bolt an enterprise’s locked-down, multi-agent, evidence-required pipeline onto a two-person prototype and you have added friction for no benefit. Run “tests pass, ship it” on a payments system and you have built an incident generator with a green checkmark on top. Most bad advice in this space is one position on that spectrum prescribing to another.

What review is actually for now

Review was built to check an author’s reasoning and catch bugs + knowledge share with the team. An agent does reason, but that reasoning is usually thrown away rather than attached to the code, so the reviewer has to reconstruct a rationale that never made it into the diff. The good news: that is a tooling problem, and capturing the reasoning makes review dramatically easier.

This is the part that genuinely changed, and I think it is underappreciated.

When a human writes code, intent comes along for free. The reasoning, the alternatives weighed and discarded, lived in the author’s head, and review was you checking that reasoning. Modern agents do reason, often visibly, producing thinking traces and weighing options and explaining themselves as they go. The catch is that this reasoning is usually discarded the moment the diff is produced. It is rarely captured, rarely attached to the PR, and in any case it is the agent’s reasoning about how to implement the task, not a human’s judgment about whether it was the right task to begin with. So review shifts from checking reasoning that sits in front of you to reconstructing intent that never got written down, which is harder and slower, and we keep acting surprised that it takes 441% longer.

A 2026 paper, AI Slop and the Software Commons, analyzed 1,154 posts across 15 Reddit and Hacker News threads where developers discussed “AI slop”. One line from a developer caught my eye: reviewing an agent’s PR made them “the first human being to ever lay eyes on this code”.

That points straight at the fix. In normal review the author already understood the change and you were checking their work. With an agent PR, nobody has reconstructed the why yet. The reviewer is the first to try.

As the paper puts it, review “wasn’t built to recover missing intent”. The encouraging part is that missing intent is recoverable: the reasoning existed, we just discarded it. Have the agent state what it was trying to do and what it ruled out, capture that as a decision log on the PR, and a large part of the reconstruction cost disappears. This is a tooling problem, and tooling problems get solved.

None of which makes “have the AI review the AI” a complete answer on its own. A second model with different priors genuinely catches real bugs, and it catches a lot of them, which is why you should run one. What it does not supply is the human judgment about whether this is the right change to build in the first place. That judgment stays with a person, and it happens to be the most interesting part of the job, the part worth keeping.

The tools are good, but not always for the reason they advertise

The current AI reviewers are genuinely good, and they occasionally don’t flag the same lines as each other, so the right move is not picking the best one but running two that are built differently.

The dedicated AI review tools are good now, and I think you should be running at least your main coding agent if not a dedicated review agent on everything, side projects included.

CodeRabbit is the most widely deployed and topped the independent Martian benchmark (January to February 2026) on F1, around 49% precision with the best recall in the field.

Greptile trades precision for recall: around an 82% bug-catch rate against CodeRabbit’s 44% in one benchmark, at the cost of more false positives.

Anthropic’s Code Review reports under 1% of its findings marked incorrect by their engineers, and the figure I would actually show a manager: it raised their internal rate of PRs receiving a substantive review from 16% to 54%. The long tail of changes that used to get a glance and an approval now gets read by something.

The most useful result I have seen this year is not from a vendor. An engineer ran four reviewers in parallel, CodeRabbit, Sentry Seer, Greptile and Cursor BugBot, across 146 real PRs and 679 findings over three and a half weeks:

Of 617 distinct flagged locations, 93.4% were caught by exactly one of the four tools. 6% by two. Almost none by three. None at all by all four.

The four tools never once flagged the same line. Each was strong at a different class of problem: Greptile with near-zero false positives on correctness and architecture, CodeRabbit with the widest net and one-click fixes, Seer best on production-failure severity. That is the adversarial review argument demonstrated on a real codebase rather than in a paper. Heterogeneity is the whole point. Four copies of one model is a single reviewer with a larger invoice, whereas four genuinely different reviewers surface a set of bugs no single member could find alone, the human included.

In practice: do not agonize over the single best tool, there isn’t one. At the high-stakes end, run two with deliberately different characters (the experiment above paired Greptile for everyday correctness with Seer for production-failure severity, with almost no overlap). If you are solo, one good reviewer plus real tests is plenty. And whatever the marketing says, measure it on your own code, because every one of these results was specific to a particular codebase, and yours will be too.

Should we just let AI review more of it?

The machine is already reviewing more of your code than you are. The only real decision left is whether you do that deliberately, and the amount of human you keep should scale with your blast radius.

I keep hearing a question that would have been heresy a year ago, now from experienced engineers: should the machine be doing more of the reviewing, perhaps most of it? I no longer think that is a foolish question.

The uncomfortable part is that AI review works. Under 1% of Anthropic’s findings are marked wrong, the tools catch bugs humans read straight past, and they do not get tired on the thirtieth PR of the day, which is exactly when a human is least reliable. Meanwhile humans are visibly not keeping up: zero-review merges are up 31% and review times are up triple digits. In a real sense the machine is already reviewing more of the code than we are. The honest framing is not “should we let AI review more” but “AI is already doing it, are we going to be deliberate about that or let it happen by default while pretending humans still read everything”.

Loop engineering sharpens this. The premise of a loop is that you stop being the person who prompts the agent and instead build a system that prompts it, and a central part of that system is a judge: an agent that decides whether the work is done before moving on. The reviewer is the next role being designed out of the inner loop, on purpose. We spent a year automating the writing, and the loops are now automating the checking, and the human keeps getting pushed up and out. “Where does the human stay” is not a seminar question, it is something you decide every time you wire up a loop, whether or not you realize you are deciding it.

Where I currently land is: the answer may not be “a human reads every line”. That is over. But it is also not “let the loop review itself and walk away”. When an agent writes the code, another reviews it and a third judges it, you have a closed loop of models with broadly correlated blind spots, especially when they come from the same family, confidently agreeing in the same places. A confident “looks good” with no human anywhere in it is borrowed confidence: the system’s certainty becomes yours, and nobody actually understood anything. The loop can be both very sure and very wrong, with no human left to tell the difference.

So the human does not leave; the human moves up a level. You stop reviewing every diff and start owning the parts that do not transfer to a model. Accountability matters.

The judgment of whether this is even the right change to build, as distinct from whether the code is correct. The high-blast-radius gates where being wrong is expensive. And the awkward one: the behavior nobody specified, because a model reviews the code that exists and rarely flags the requirement that nobody thought to write down, which remains a human-shaped gap I do not expect to close soon.

Human in the loop becomes human on the loop: sampling, spot-checking and auditing the system rather than reading every PR, and spending your limited attention where being wrong would actually hurt.

This is already how I work on my own projects, including the open-source ones that now see more PRs in a day than I could carefully read in an evening. I point Claude Code or Codex at a batch of incoming PRs and ask for a first pass: a high-level read of what looks safe to merge, what needs more work, and what is genuinely high-risk. I do not auto-merge on the result, and I do not lazy-merge whatever it approves. What it gives me is a way to allocate attention. I can spend a few minutes confirming the changes it considers low-risk, and put real, careful time into the ones it flags as dangerous. The detail that matters is that this is not my old review hour made slightly faster. It is a different shape of hour, and at the volume I now deal with, it is the main reason the queue stays survivable at all.

📷 Codex and Claude Code giving me a first-pass, risk-sorted read of a batch of PRs. The triage is the help. The merge decision stays mine.

A more extreme version of the same move is Kun Chen, an ex-Meta L8 engineer now shipping around 40 PRs a day as a solo builder, who has largely stopped reviewing code as told to . It would be easy to dismiss this, except he is an L8, unusually good at the thing he stopped doing, which is what makes it interesting. He runs 20 to 30 agents in parallel and has moved his effort into the plan: he writes detailed plans up front, the agents run for hours against them, and he says plan quality determines how long they can run unattended. That is the move I described above. It is worth being precise about what actually happened, because it is not that he stopped verifying. The intent did not vanish, he wrote it down himself in the plan, so the “first human to ever lay eyes on this” problem is half-solved: a human did understand the why, just up front rather than after. And he did not work without a net, he built an automated review gate (he calls it No Mistakes) that checks the code before it merges, and he stays on escalation when an agent gets stuck. The human does the expensive thinking before the code exists and the machine does the line-by-line afterward, which may well be the shape of where this goes.

But he is a solo builder with no large team and no decade-old system full of landmines beneath him. The exact conditions that make 40 PRs a day without review rational for him are conditions most readers do not have. Copy his workflow onto a team shipping to many users and you reproduce the Faros numbers on your own dashboard. He is not wrong; he is a long way down one specific end of the spectrum.

Which is the spectrum point again. Solo with no users: letting AI review almost all of it is a defensible 2026 position, and you should not feel guilty about it. Maintaining something large for many people: let the machine handle the first pass, the second pass and the boring 90%, but keep a real human on the load-bearing paths and do not let the loop close completely on anything that can hurt someone. How much human you keep is a dial, and you set it by blast radius, not by guilt.

What to actually do

Stop reviewing everything to the same depth. Spend scarce human attention only where being wrong is costly, and let cheap deterministic gates and AI reviewers handle the rest.

The organizing idea is to match review effort to the cost of being wrong, push the cheap deterministic work as early as possible, and reserve human attention for what only humans can do.

Tier by risk, not by author. A config change earns a linter and a glance. A revision to your core business logic path earns the full stack: types, tests, two different AI reviewers, a human who owns that system, and a security pass. Do not spend a heavy review on boilerplate, and do not wave through a big change because the tests are green. The layered approach is the same everywhere; what changes is how many layers a given diff has to clear.

Fast-fail the expensive tail. The most useful recent finding for teams drowning in agent PRs is Early-Stage Prediction of Review Effort (January 2026), which studied 33,707 agent-authored PRs. Agents are good at small, well-defined changes, around 28% merge almost instantly, but they tend to “ghost” the moment they get subjective feedback, abandoning the back-and-forth that review actually is. (A companion 2026 paper found reviewer abandonment accounted for 38% of rejected agent PRs.) The researchers built a “circuit breaker” that predicts high-maintenance PRs from cheap signals like file types and patch size before a human looks, and it works well. Triage agent PRs up front, fast-track the trivial ones, and do not let a person sink an hour into a sprawling change the agent will abandon as soon as you push back.

Raise the bar for what you will even review. The fix for being buried is not locking down the repository, it is refusing to review changes that arrive without evidence. Require, before review: a statement of what the change is for, a diff that is not 3,500 lines with no comments, the test output, and proof it was actually run. This is how you stop being the first human to read the code. You push the intent-reconstruction work back onto whoever submitted it, where it is cheap, rather than absorbing it yourself, where it is expensive.

Keep PRs small, deliberately. Agent PRs run large, 51% larger on average in the Faros data, and reviewer engagement is one of the strongest predictors that a PR merges at all. A large unreviewable PR gets rejected outright or, worse, rubber-stamped. Instruct your agents to produce small commits. A diff a human can actually read is now a design constraint, not a courtesy.

Read the test changes more carefully than the code. This is the agent failure mode to watch. The agent changes behavior, then “fixes” the test by rewriting the assertion to match the new, broken behavior. A green check over 200 edited tests means nothing until you have confirmed the edits were correct. Treat any diff that rewrites many tests as a flag and read those first. Mutation testing earns its place here: coverage tells you a line ran, mutation testing tells you whether the test would notice if that line were wrong.

Treat CI as the wall that does not move. Watch for the patterns GitHub now warns reviewers about: removed tests, skipped lint, lowered coverage thresholds, a duplicated helper that already exists elsewhere, and untrusted input flowing into a prompt. That last one deserves emphasis, because agent-built features are a fresh source of prompt injection: if a change pipes user-controlled text into an LLM call without thinking about what that text can instruct the model to do, the vulnerability is not visible in the diff, it is latent in the data that will arrive later. Agents will also weaken CI to make themselves pass, not maliciously, just gradient descent finding the cheapest path to green. Deterministic gates are the one part of the pipeline that cannot be talked out of their verdict by a confident paragraph, so keep them strict.

A human owns the merge. A model cannot be paged and cannot be held responsible for what it shipped, so whoever clicks merge owns it. When an AI review says “looks good” in a calm, confident voice, it is handing you confidence it has not necessarily earned. Treat every AI review as a sensor, not a verdict: data, not a decision.

If you are solo with no users, the tiering, the test-change discipline and CI are most of what you need; the rest is overhead until people show up. If you are the large organization, all of it is the baseline, and the triage and intake bar are the difference between a review process that scales and one that quietly collapses.

What this means if you run a team

The bottleneck is no longer how fast you write code, it is how fast a trusted human can be confident in a review. Cutting the people who provide that confidence because “AI made us faster” simply converts the saving into future incidents.

The binding constraint on shipping is no longer how fast you can write code. It is how fast a trusted human can be confident a change is correct. Any plan that treats generation as the bottleneck and review as free will quietly stall, with the velocity dashboard staying green the whole way.

The Faros report is direct about this: QA and review work rises even as output rises, so reducing engineering headcount because “AI made us faster” is dangerous unless you have closed the review gap first. The senior-engineer tax, review time up by triple digits, falls hardest on the people you can least afford to bottleneck, and it is invisible to any metric that only counts merged PRs.

Open source maintainers hit this wall first and hardest. The steady stream of plausible but hollow contributions costs real triage time even when it is well-intentioned, and that is the canary. Companies are next. The ones handling it well treat review capacity as a real resource to be measured, protected and spent deliberately, not as slack that AI has freed up.

Writing got cheap, understanding didn’t

Code review did not become less important when agents arrived. It became the central activity. Writing code is increasingly solved and getting cheaper by the month; the durable advantage is the system that lets you trust what was written.

Do not take the one-size answer in either direction. If you are solo with no users, the enterprise horror stories about churn and duplication are a future risk, not today’s fire, so lean on your tests, review what matters, and stay honest that the deferred work is still owed. If you maintain something large for many people, every alarming number here is about you, and the only thing that holds is a tiered, evidence-required, deliberately heterogeneous review process with a human owning the merge.

What is constant across the whole spectrum is the underlying economics. We made writing cheap, and understanding stayed exactly as expensive as it has always been. The teams that do well over the next few years will not be the ones generating the most code, they will be the ones who built a review system they can actually trust, and who never confuse “the tests passed” with “a person understands what this does and why”.

Or, as keeps putting it, your job is to deliver code you have proven to work. Agents have not changed that. They have made the proving the center of the job rather than an afterthought, and I think that is a good trade.

Understanding a system well enough to stand behind it is the most durable and most interesting skill in software, and there has never been a better time to get extraordinarily good at it.

I believe this may be the future of how we work with coding agents. However, its still early, I’m skeptical and you absolutely have to be careful about token costs (usage patterns can vary wildly if you are token rich or poor), so I want to unpack what it is and what it means.

Peter Steinberger recently said: “You shouldn’t be prompting coding agents anymore. You should be designing loops that prompt your agents.” Similarly, Boris Cherny, head of Claude Code at Anthropic, said “I don’t prompt Claude anymore. I have loops running that prompt Claude and figuring out what to do. My job is to write loops”.

Okay, so what does any of that mean?

For like two years the way you got something out of a coding agent was you wrote a good prompt and shared enough context. You type a thing, you read what came back, you type the next thing. The agent is a tool and you are holding it the entire time, one turn after the other. That part is kind of over, or at least some think it’s going to be.

Now you build a small system that finds the work, hands it out, checks it, writes down what is done and then decides the next thing, and you let that system poke the agents instead of you. I wrote before about the cousin of this, agent harness engineering, which is making the environment one single agent runs inside and the factory model - the system that builds the software. Loop engineering sits one floor above the harness. The harness but it runs on a timer, it spawns little helpers, and it feeds itself.

The thing that surprised me is this is not really a tool thing anymore. A year ago if you wanted a loop you wrote a pile of bash and you maintained that pile forever and it was yours and only yours. Now the pieces just ship inside the products. Steinberger’s list maps almost exactly onto the Codex app, and then almost the same onto Claude Code. And once you notice the shape is the same you stop arguing about which tool, you just design a loop that still works no matter which one you happen to be sitting in.

The five pieces, and then notes

A loop needs five things and then one place to remember stuff. Let me list it first and then map it.

1. Automations that go off on a schedule and do discovery and triage by themselves.

2. Worktrees so two agents working in parallel dont step on each other.

3. Skills to write down the project knowledge the agent would otherwise just guess.

4. Plugins and connectors to plug the agent into the tools you already use.

5. Sub-agents so one of them has the idea and a different one checks it.

Then the sixth thing, the memory. A markdown file, or a Linear board, anything that lives outside the single conversation and holds what’s done and what is next. Sounds too dumb to matter. But it’s the same trick every long running agent depends on and I went into it in long-running agents, the model forgets everything between runs so the memory has to be on disk and not in the context. The agent forgets, the repo doesnt.

Both products have all five now.

The names are a bit different here and there but the capability is the same thing. Let me go one by one because honestly the details are where a loop either holds together or quietly leaks everywhere.

Automations, this is the heartbeat

Automations are what make a loop an actual loop and not just one run you did once. In the Codex app you make one in the Automations tab and you pick the project, the prompt it will run, how often, and if it runs on your local checkout or on a background worktree. The runs that find something go to a Triage inbox, and the runs that find nothing just archive themselves which is nice. OpenAI uses them internally for boring stuff like daily issue triage, summarizing CI failures, writing commit briefings, hunting bugs somebody added last week. And an automation can call a skill, so you keep the recurring thing maintainable, you fire $skill-name instead of pasting a giant wall of instructions into a schedule that nobody will ever update.

Claude Code gets to the same place but through scheduling and hooks. You can run a prompt or a command on a interval with /loop, you can schedule a cron task, you can fire shell commands at certain points in the agent lifecycle with hooks, or you push the whole thing to GitHub Actions if you want it to keep running after you close the laptop. Same idea exactly, you define an autonomous task, you give it a cadence, and the findings come to you so you are not the one going around checking.

There is a second in-session primitive worth knowing, and it's the one closer to what this whole post is about. /loop re-runs on a cadence. /goal keeps going until a condition you wrote is actually true, and after every turn a separate small model checks whether you are done, so the agent that wrote the code isnt the one grading it. You give it something like "all tests in test/auth pass and lint is clean" and walk away. Codex has the same thing, also called /goal, it keeps working across turns until a verifiable stopping condition holds, with pause and resume and clear. Same primitive, both tools, wich is kind of the pattern for this whole article.

So this is the part that surfaces the work. The rest of the loop is what acts on it.

Worktrees so parallel doesn’t turn into chaos

The second you run more than one agent the files start colliding, that becomes the failure. Two agents writing the same file is the exact same headache as two engineers committing to the same lines and nobody talked to each other first. A git worktree fixes it, its a separate working directory on its own branch sharing the same repo history, so one agent’s edits literally can not touch the other one’s checkout.

Codex builds the worktree support right in so several threads hit the same repo at once and dont bump into each other. Claude Code gives you the same isolation with git worktree, a --worktree flag to open a session in its own checkout, and a isolation: worktree setting you stick on a subagent so each helper gets a fresh checkout that cleans itself up after. I wrote about the human side of all this in the orchestration tax, the worktrees take away the mechanical collision but YOU are still the ceiling, your review bandwith decides how many you can actually run, not the tool.

Skills, so you stop explaining your project every single time

A skill is how you stop re-explaining the same project context every session like a goldfish. Both tools use the same format, a folder with a SKILL.md inside holding instructions and metadata, and then optional scripts, references, assets. Codex runs a skill when you call it with $ or /skills, or by itself when your task matches the skill description, which is the reason a tight boring description beats a clever one. Claude Code does it the same way and I wrote the pattern up in agent skills.

Skills are also where intent stops costing you over and over. I argued in the intent debt that an agent starts every session cold and it will fill any hole in your intent with a confident guess. A skill is that intent written down on the outside, the conventions, the build steps, the “we dont do it like this because of that one incident”, written one time where the agent reads it every run. Without skills the loop re-derives your whole project from zero every cycle, with skills it kind of compounds.

One thing to keep straight, the skill is the authoring format and a plugin is how you ship it. When you want to share a skill across repos or bundle a few together you package them as a plugin. True in Codex, true in Claude Code.

Plugins and connectors, the loop touches your real tools

A loop that can only see the filesystem is a tiny loop. Connectors, which are built on MCP, let the agent read your issue tracker, query a database, hit a staging api, drop a message in Slack. Codex and Claude Code both speak MCP so the connector you wrote for one usually just works in the other. And plugins bundle connectors and skills together so your teammate installs your setup in one go instead of rebuilding the whole thing from memory.

This is the difference between an agent that says “here is the fix” and a loop that opens the PR, links the Linear ticket and pings the channel once CI is green by itself. The connectors are the reason the loop can act inside your actual environment instead of just telling you what it would do if it could.

Sub-agents, keep the maker away from the checker

The most useful structural thing in a loop, by far, is splitting the one who writes from the one who checks. The model that wrote the code is way too nice grading its own homework. A second agent with different instructions and sometimes a different model catches the stuff the first one talked itself into.

Codex only spawns subagents when you ask, runs them at the same time and then folds the results back into one answer. You define your own agents as TOML files in .codex/agents/, each with a name, a description, instructions and optional model and reasoning effort, so your security reviewer can be a strong model on high effort while your explorer is some fast read-only thing. Claude Code does the same with subagents in .claude/agents/ and agent teams that pass work between them. The usual split in both is one agent explores, one implements, one verifies against the spec.

I made this case twice already, once as the code agent orchestra and once as adversarial code review. The reason it matters specifically inside a loop is the loop runs while you are not watching, so a verifier you actually trust is the only reason you can walk away. Subagents do burn more tokens since each one does its own model and tool work, so spend them where a second opinion is worth paying for. This is also basically what Claude Code’s /goal does under the hood, a fresh model decides if the loop is done instead of the one that did the work, the maker and checker split applied to the stop condition itself.

What one loop looks like

Stick it together and a single thread turns into a little control panel. Here is one shape I keep using.

An automation runs every morning on the repo. Its prompt calls a triage skill that reads yesterdays CI failures, the open issues, the recent commits, and writes the findings into a markdown file or a Linear board. For each finding that is worth doing the thread opens an isolated worktree and sends a sub-agent to draft the fix, and a second sub-agent reviews that draft against the project skills and the existing tests.

Connectors let the loop open the PR and update the ticket. Anything the loop can not handle lands in the triage inbox for me. The state file is the spine of the whole thing, it remembers what got tried, what passed, what is still open, so tomorrow morning the run picks up where today stopped.

And look at what you actually did there. You designed it one time. You did not prompt any of those steps. Thats Steinberger’s whole point made real, and its the same loop in Codex or in Claude Code because the pieces are the same pieces.

What the loop still does not do for you

The loop changes the work, it does not delete you from it. And three problems actually get sharper as the loop gets better, not easier.

Verification is still on you. A loop running unattended is also a loop making mistakes unattended. The whole reason you split the verifier sub-agent from the maker is to make the loop’s “its done” mean something, and even then “done” is a claim and not a proof. I keep saying the same line from code review in the age of AI, your job is to ship code you confirmed works.

Your understanding still rots if you allow it. The faster the loop ships code you did not write, the bigger the gap between what exists and what you actually get. Thats comprehension debt and a smooth loop just makes it grow faster unless you read what the loop made.

And the comfortable posture is the dangerous one. When the loop runs itself its very tempting to stop having an opinion and just take whatever it gives back. I called that cognitive surrender. Designing the loop is the cure when you do it with judgement and the accelerant when you do it to avoid thinking, same action, opposite result.

Build the loop. Stay the engineer.

I think this is a preview of how our work is going to evolve. That said, If I weren’t reviewing the code myself or if I relied entirely on automated loops to fix it my product’s quality would suffer. I’d likely end up stuck in a downward spiral, continuously digging myself into a deeper hole.

That said, go ahead and set up your loops, but don’t forget that prompting your agents directly is also effective. It’s all about finding the right balance.

Loops can also result in different outcomes depending on you. Two people can build the exact same loop and get completely opposite results. One uses it to move faster on work they understand deeply. The other uses it to avoid understanding the work at all. The loop doesn’t know the difference. You do.

That’s what makes loop design harder than prompt engineering, not easier. Cherny’s point isn’t that the work got easier. It’s that the leverage point moved.

Build the loop. But build it like someone who intends to stay the engineer, not just the person who presses go.

A long-running AI agent can keep making progress over hours, days, or weeks. It can do this across many context windows and sandboxes, recover from failure, leave structured artifacts behind, and resume where it left off.

For two years the dominant image of an “AI agent” has been a chat window with a clever loop in it. You type a goal, the agent calls some tools, you watch tokens stream by, you stop watching when the work runs out of patience or the context window fills up. That paradigm got us a long way, but it has a ceiling. The model forgets. It declares “task complete” when it isn’t. It re-introduces a bug it fixed nine turns ago. The whole thing is structured around a single sitting.

Long-running agents are what comes next. The idea is easy to state: an agent that keeps making forward progress on a goal across many sessions and many sandboxes, possibly many days or weeks, while leaving the workspace clean enough that the next session can pick up where the last one left off. The engineering is harder. You have to solve for persistence, recovery, and verification in a way that doesn’t just paper over the cracks. You have to build a state layer that lives outside the model’s context window, and you have to design the handoff between sessions so the agent doesn’t lose its mind when it wakes up and finds itself in a different sandbox with a different context window.

This post is my attempt to lay out what’s changed, who’s pushing on it, and how an engineer can use long-running agents today without writing the whole thing from scratch.

What “long-running” actually means

“Long-running” gets used to mean at least three different things in practice, and it helps to keep them separate.

Long-horizon reasoning. The agent has to plan and execute over many dependent steps. This is mostly a model-quality story: coherence, planning, the ability to recover from a wrong turn ten steps ago. METR has been tracking this with their time horizon metric, which estimates how long a task a frontier model can complete with 50% reliability. The headline finding is that the metric has been doubling roughly every seven months since 2019, and their TH1.1 update earlier this year doubled the count of 8-hour-plus tasks in the eval set. If that curve holds, frontier agents complete tasks at the day scale by 2028 and the year scale by 2034.

Long-running execution. The agent’s process runs for hours or days. Maybe it’s a coding job, maybe it’s a research sweep, maybe it’s a 24/7 monitoring service. The model might be invoked thousands of times across the run. This is mostly a harness story, and it’s the one this post is mostly about.

Persistent agency. The agent has an identity that outlives any single task. It accumulates memory, learns user preferences, and is always available. This is the Memory Bank flavor of long-running.

In practice the three blur together. A real production agent does long-horizon reasoning inside a long-running execution backed by persistent agency. But the engineering problems are different in each, and so are the products that solve them.

Why this matters

There are two reasons I believe this work matters a lot right now.

The first is a phase change in what’s economically feasible to delegate. An agent that runs for ten minutes can answer a question, summarize a doc, fix a small bug. An agent that runs for ten hours can own an entire feature, finish a migration that was on the backlog for six quarters, or do the kind of overnight research sweep that used to require a junior analyst. One of Anthropic’s Claude Sonnet announcements put concrete numbers on this last fall: 30+ hours of autonomous coding in internal tests, including one run that produced an 11,000-line Slack-style app. That’s already past the threshold where the answer to “should I delegate this?” is no longer obvious.

The second is that persistence changes what the agent is. A stateless agent answers your question and disappears. A long-running one accumulates context: which competitor moved which way last week, which test flaked twice on Tuesday, what you usually mean by “the dashboard.” Anthropic’s Project Vend was the most public early demonstration of this. They had a Claude instance run an actual office vending business for a month, managing inventory, setting prices, talking to suppliers. It failed in informative ways, and the second phase ran much better, but the point wasn’t profitability. The point was watching what kinds of weird coherence problems show up when an agent has to maintain identity across weeks instead of turns.

Those are the same problems every team building production agents now hits.

The three walls every long-running agent hits

Three walls show up in basically every write-up I’ve read this year.

Finite context. Even a 1M-token window fills. And context rot, the steady degradation of model performance as the window gets full, kicks in well before the hard limit. A 24-hour run is not going to fit in any context window the field has on its roadmap. Something has to give.

No persistent state. A new session starts blank. Anthropic’s framing in their scientific computing post is the cleanest version I’ve seen: “imagine a software project staffed by engineers working in shifts, where each new engineer arrives with no memory of what happened on the previous shift.” Without an explicit persistence story, every shift change is a productivity disaster.

No self-verification. Models reliably skew positive when they grade their own work. Asked “are you done?” they answer “yes” more often than they should. Without a separate signal that the work meets a bar, you get the agent that ships at 30% complete with full confidence.

Long-running agent designs are mostly answers to these three problems. The major labs have converged on similar shapes of answer, but with very different surface area.

The Ralph loop: one of the simpler practitioner versions of long-running agents

The Ralph loop (sometimes called the Ralph Wiggum technique) is one of “simpler” practitioner version of long-running agents, popularized by Geoffrey Huntley and Ryan Carson. The reference implementation is literally a bash script that loops:

1. Pick the next unfinished task from a list (prd.json or equivalent).

2. Build a prompt with the task, the relevant context, and any persistent notes.

3. Call the agent.

4. Run tests or other checks.

5. Append what happened to progress.txt.

6. Update the task list (done, failed, blocked).

7. Go back to step 1.

The reason it works is the same reason any of the harnesses below work: state lives outside the agent’s context. prd.json is the plan, progress.txt is the lab notes, AGENTS.md is the rolling rulebook. The agent itself is amnesiac, but the filesystem isn’t. Each iteration starts fresh and reads enough state from disk to keep going. Carson’s Compound Product extends the idea by chaining multiple loops (an analysis loop that reads daily reports, a planning loop that emits a PRD, an execution loop that writes the code), which is roughly the open-source version of the planner-generator-evaluator triad Anthropic landed on independently.

I went deeper on all of this in Self-improving agents: task list structure, progress files, QA gates, monitoring, the failure modes you’ll actually hit. The short version is that you can build a working long-running agent in an evening with a bash script and a JSON file. Most of what Google and Anthropic have productized is the work of making this pattern recoverable, secure, and observable at scale.

The big-lab stories below are different ways of paying for that production-readiness.

Anthropic: harnesses, then the brain/hands/session split

Anthropic has been the most public about the engineering. Two posts are worth reading end-to-end.

The first is “Effective harnesses for long-running agents”, which lays out a two-agent harness for autonomous full-stack development. An initializer agent runs once at the start of a project to set up the environment, expand the prompt into a structured feature-list.json, and write an init.sh that future sessions will run on boot. A coding agent is then woken up over and over, each session asked to make incremental progress on one feature, run tests, leave a claude-progress.txt note, and commit. A test ratchet (“it is unacceptable to remove or edit tests because this could lead to missing or buggy functionality”) sits in the prompt to stop the very common failure of an agent deleting failing tests to “make them pass.” InfoQ’s writeup extends this into a planner, generator, and evaluator triad, on the same logic that separating generation from evaluation matters because models grade their own work too generously.

The second is “Scaling Managed Agents: Decoupling the brain from the hands”, the architectural post behind Claude Managed Agents (Anthropic’s hosted runtime, launched in early April). The argument is that an agent has three components that should be independently replaceable. The Brain is the model and the harness loop that calls it. The Hands are sandboxed, ephemeral execution environments where tools actually run. The Session is an append-only event log of every thought, tool call, and observation.

This sounds abstract and it isn’t. Anthropic’s framing: “every component in a harness encodes an assumption about what the model can’t do on its own.” When you couple them, an assumption that goes stale (e.g., the model used to need an explicit planner and now plans natively) means the whole system has to change at once. When you decouple them, the harness becomes stateless, sandboxes become cattle, not pets, and a brain crash doesn’t lose the run. A fresh container calls wake(sessionId) and reconstitutes the state from the log. They reported time-to-first-token dropped ~60% at p50 and over 90% at p95 just from being able to start inference before the sandbox is ready.

The session-as-event-log idea is the part most teams underappreciate. It is what makes a long-running agent recoverable. Without it, a container failure is a session failure and you’re debugging into a stale snapshot. With it, the agent’s memory is a queryable artifact that lives outside whatever process happens to be running at the moment.

For the scientific computing crowd, Anthropic’s long-running Claude post reduces all of this to a simpler stack: CLAUDE.md as a living plan the agent edits as it learns, CHANGELOG.md as portable lab notes, tmux plus SLURM plus git as the execution and coordination layer, and the Ralph loop, a for loop that kicks the agent back into context whenever it claims completion and asks if it’s really done. Their flagship case study is a Boltzmann solver Claude Opus built over a few days that reached sub-percent agreement with a reference CLASS implementation. Months-to-years of researcher time, compressed.

Same patterns across all three posts: an explicit plan file, an explicit progress file, structured handoffs between sessions, separate generation from evaluation, and a loop that refuses to let the agent stop early.

Cursor: planners, workers, judges

Cursor’s “Scaling long-running autonomous coding” is the other essential read this year. They walked into walls that Anthropic mostly papered over.

Their first attempt was a flat coordination model: equal-status agents writing to shared files with locks. It became a bottleneck and made the agents risk-averse, churning rather than committing. Their second attempt swapped locks for optimistic concurrency control, which removed the bottleneck but didn’t fix the coordination problem. The third design is what’s running in production now and what they describe as solving most of the problem:

• Planners continuously explore the codebase and emit tasks. They can recursively spawn sub-planners.

• Workers are focused executors. They don’t coordinate with each other and they don’t worry about the big picture.

• Judges decide when an iteration is finished and when to restart.

Two things stand out from the post. One: “a surprising amount of the system’s behavior comes down to how we prompt the agents” more than the harness or the model. Two: different models slot into different roles. Their reported finding is that a GPT model was better than Opus for extended autonomous work specifically because Opus tended to stop early and take shortcuts. Same task, different role, different model. The matching is becoming part of the design surface.

This pairs with Composer 2 (their proprietary frontier coding model that ships in Cursor 3) and their background cloud agents: long-running tasks that run on Anysphere’s cloud infrastructure rather than your laptop. Eight-hour refactors and codebase-wide migrations survive a closed lid. You can start a task locally, hit run in cloud when you realize it’ll take 30 minutes, and re-attach later from your phone. Each agent runs in an isolated git worktree and merges back via PR. The handoff between local and remote is the part most teams haven’t figured out yet, and Cursor’s bet is that it has to be its own product surface.

The shape ends up close to Anthropic’s: roles are split, sessions are durable, judges sit beside the worker, and a long task runs in a cloud sandbox with git as the coordination substrate.

Google: long-running agents on the Agent Platform

Google’s announcement at Cloud Next ‘26 two weeks ago folded Vertex AI into the Gemini Enterprise Agent Platform and turned long-running agents into a named product, with named SLAs.

The pieces that matter for this post:

• Agent Runtime supports agents that “run autonomously for days at a time” with sub-second cold starts and on-demand sandbox provisioning. The launch post’s example use case is a sales prospecting sequence that takes a week to play out, which is roughly the right shape for it.

• Agent Sessions persist conversation and event history. You can pin them to a custom session ID that maps to your own CRM or DB record, so the agent’s state lives next to the business state instead of in a separate AI silo.

• Agent Memory Bank is the persistent long-term memory layer, generally available as of Next ‘26. It curates memories from sessions, scopes them to a user identity, and exposes a search API so the next agent invocation can pull what’s relevant. Payhawk reported that auto-submitting expenses through a Memory-Bank-backed agent cut submission time by over 50%.

• Agent Sandbox handles hardened code execution.

• Agent-to-Agent Orchestration, Agent Registry, Agent Identity, Agent Gateway, Agent Observability, and Agent Simulation cover basically every operational concern you’d otherwise build by hand for a production fleet, including the cryptographic-identity-and-audit-log story enterprises actually need to ship.

Architecturally this is the same brain/hands/session split Anthropic described, just productized at platform scale and bundled with ADK (the code-first dev kit) and Agent Studio (the visual one). If you’re building inside Google Cloud, you don’t have to design a session log or a memory store from scratch anymore. You wire an ADK agent into Memory Bank and Sessions, deploy onto Agent Runtime, and the persistence question is answered.

Notice how much this looks like the pattern Anthropic and Cursor describe, just unbundled into named services with SLAs. Three years ago you’d have built all of this yourself. Now you pick which version of “decoupled brain, hands, and session” you want to rent.

Five patterns for long-running agents in production

Shubham Saboo and I wrote up five design patterns we’ve seen separate working long-running agents from demos. They aren’t Google-specific, but they map cleanly onto the primitives Agent Runtime now exposes, so it’s worth walking through them here in shortened form.

Checkpoint-and-resume. The most common multi-day failure is context loss. An agent processes 200 documents over four hours, hits an error on document 201, and without a checkpoint you start from scratch. Treat the agent like a long-running server process: write intermediate state to disk, checkpoint every N units of work, recover from failures. The Agent Runtime sandbox gives you a persistent filesystem, but choosing the right checkpoint granularity (not every step, not only the end) is on you.

Delegated approval (human-in-the-loop). Most “human-in-the-loop” implementations are: serialize state to JSON, fire a webhook, hope someone responds. The state goes stale, the notification gets buried, the agent re-deserializes into a slightly different world. Long-running runtimes let the agent pause in place with full execution state intact: reasoning chain, working memory, tool history, pending action. Hours of human time pass, the agent consumes zero compute, and it resumes with sub-second latency. Mission Control is Google’s inbox for this. The pattern works regardless of vendor.

Memory-layered context. A seven-day agent needs more than session state. Memory Bank handles long-term curated memory, Memory Profiles add low-latency lookups, and the failure mode you’ll hit in production is memory drift: the agent learns a procedural shortcut from a few atypical interactions and starts applying it broadly. Govern memory like you govern microservices. Agent Identity controls who can read and write which banks. Agent Registry tracks which version of which agent is running. Agent Gateway enforces policy on the wire. The auditing question stops being “what are my agents doing?” and becomes “what are my agents remembering, and how is that changing their behavior?”

Ambient processing. Not every long-running agent talks to a human. Some sit on a Pub/Sub stream or a BigQuery table and act on events as they arrive: content moderation, anomaly detection, inbox triage. The architectural decision worth making early is to not hardcode policy into the agent. Define it in the Gateway and the fleet picks up policy changes without redeploys. Ambient agents run unsupervised for long stretches, and the only sane way to update a hundred of them is to update the policy layer once.

Fleet orchestration. In real systems, you rarely have one agent. A coordinator delegates sub-tasks to specialists (a Lead Researcher Agent, a Scoring Agent, an Outreach Agent), each running independently for different durations. Each specialist gets its own Identity (so the Outreach Agent can’t read financial data meant for Scoring), its own policy enforcement, its own Registry entry. This is the same coordinator/worker shape distributed systems have used for decades. What’s new is that ADK handles it declaratively with graph-based workflows, and a bad deployment in one specialist doesn’t cascade to the others.

The patterns compose. A compliance system might use checkpointing for document processing, delegated approval for review gates, memory layering for cross-session knowledge, and fleet orchestration to coordinate the specialists. The opening question is always the same: what’s the longest uninterrupted unit of work your agent needs to perform? Minutes, and you don’t need long-running agents. Hours or days, and these patterns are where to start. The full write-up with code samples covers each pattern in depth.

So how do you actually build one today?

This is the practical question and it has a different answer depending on what you’re building.

You’re a developer who wants long-running coding work on your own repo. Just use Claude Code (or Antigravity, Cursor, or Codex). The harness is already there. Treat your AGENTS.md like a pilot’s checklist: short, every line earned by a real failure. Add hooks for typecheck and lint that surface failures back to the agent. Write a plan file before the agent starts. Use the Ralph loop when the agent claims it’s done and you don’t believe it. For multi-hour or overnight jobs, run in a worktree so a closed laptop doesn’t kill the run, and have it commit progress every meaningful unit of work. This is the path most people should take, and it’s where the most leverage is right now.

You’re building a hosted agent product. Don’t build the runtime. Pick a managed one. The three real options today: Google’s Agent Platform (Agent Engine + Memory Bank + Sessions), Claude Managed Agents, or roll something on top of ADK, the Claude Agent SDK, or Codex SDK and host it yourself. The trade-off is the usual one. Managed gets you the brain/hands/session split, observability, identity, and an audit trail out of the box. Self-hosted gets you control and the ability to use weird models for weird roles (Cursor’s pattern). For most teams, the right starting point is a managed runtime plus your own ADK or SDK code for the actual loop.

You’re doing something autonomous and operational (monitoring, research, ops). Memory Bank-style persistence is what you want, and it’s the part that doesn’t exist in Claude Code. ADK + Memory Bank + Cloud Run + Cloud Scheduler is the cleanest stack I’ve seen for “agent runs every N hours, accumulates state, alerts on a threshold.” This is also where Cursor’s planner/worker/judge split starts to matter more than it does for IDE coding, because the work is genuinely parallel and the failure modes are different.

A few things matter regardless of which path you take.

Write down the done-condition before the agent starts. This is the single highest-leverage move for long runs. The Anthropic harness post calls it the feature list; Cursor calls it the planner’s task spec. Either way, it’s an external file with explicit, testable completion criteria, and it exists so the agent can’t quietly redefine done mid-run.

Separate the evaluator from the generator. Self-grading is the failure mode. A planner / worker / judge pipeline, or a generator / evaluator pair, is a real architectural pattern not a stylistic preference. Even if it’s the same model in different roles with different prompts.

Invest in the session log, not just the prompt. The append-only event log is what makes the agent recoverable, debuggable, and auditable. If you can’t reconstruct what the agent did in the last 24 hours from durable storage, what you have is a long-running shell script that happens to call an LLM, not a long-running agent.

Treat compaction and context resets as first-class. Anthropic is explicit that summarization-as-compaction wasn’t enough for very long jobs; they had to do full context resets where the harness tears the session down and rebuilds it from a structured handoff file. It is essentially how humans onboard a new engineer.

There are some real limitations right now

A few things are still genuinely unsolved.

Cost. A 24-hour run with a frontier model and a few tools is not cheap. Without budgets, circuit breakers, and a hard cap on tool spend, an agent can quietly burn through a week’s API budget in an afternoon. This is solvable, but it’s an explicit step you have to take.

Security. A long-running agent with API keys, cloud access, and the ability to run shell commands has a much larger attack surface than a chat session. The brain/hands separation pattern matters here too: credentials should be unreachable from the sandbox where model-generated code runs, which is one of the benefits Anthropic calls out for Managed Agents.

Alignment drift. Over many context windows, agents drift. The original goal gets summarized, then re-summarized, then loses fidelity. This is the part hooks and judges exist to defend against. It is also the most common reason “the agent went off and did something I didn’t ask for.”

Verification. Auditing 24 hours of autonomous activity is a real human-time problem. Observability and structured artifacts (PRs, commits, briefings, test runs) are how you make this tractable. Without them, you’re scrolling logs and you’ll miss what matters.

The human role. Defining work crisply enough that an agent can run for a day on it is harder than doing the work yourself. The skill that’s appreciating in value isn’t writing code. It’s writing specs that survive contact with an autonomous executor.

Where this is going

Google, Anthropic, and Cursor have converged on roughly the same shape. Separate the model loop from the execution sandbox from the durable session log. Split planning from generation from evaluation. Bake in compaction, hooks, and context resets. Expose memory as a managed service that any agent invocation can query.

Surface area is what differs. Google’s Agent Platform is the enterprise-stack version, with the identity and audit trail story baked in. The patterns underneath are the same. Claude Managed Agents is “Anthropic’s harness, hosted.” Cursor’s background agents are “long-running coding, pulled out of the IDE and into the cloud.”

The harder problems for the next year aren’t in any of those layers individually. They’re in the coordination above them. Many long-running agents on a shared codebase. Agents that read their own traces and patch their own harnesses. Harnesses that assemble tools and context just-in-time for a task instead of being pre-configured at startup. That’s where the agent stops looking like a smarter chat window and starts looking like a colleague who’s been on the project longer than you have.

The model is still load-bearing. But the gap between a chat window and an agent you can leave running overnight is mostly in the state, sessions, and structured handoffs wrapped around it. That’s where I’d spend my learning time right now.

You might be interested in checking out some of my O’Reilly books such as Beyond Vibe Coding, The Effective Software Engineer or Web Perf engineering in the age of AI.

If you’ve spent the last 18 months putting agents into production, you already know the models and tools have gotten dramatically better. You also know the problems that are still burning your on-call rotation are not problems you can prompt your way out of. We are running into a stack ceiling, and it is quietly creating a governance and reliability gap that the next generation of agentic systems cannot grow through.

Right now the industry is living with what I’d call excessive agency: autonomous systems given broad permissions to get things done, then left to discover - at runtime, in production - that a schema drifted, an API changed, or a downstream service started returning PII it wasn’t supposed to. Agents mark tasks “complete” while leaving a trail of corrupted state behind them. The humans find out on Monday.

This is not a failure of the people building agents. It is a failure of the stack they’re building on.

Here are the four architectural bets I think every serious team has to make in the next twelve months.

1) Agents need identities, not shared credentials

Every engineer who has shipped agents to production knows this specific flavor of dread: you have agents doing useful work, and effectively zero visibility into which tools they touched, which data they moved, or which credentials they used to do it. I call this governance debt - the silent accumulation of security and audit risk that eventually forces a full rewrite, usually right after the first incident that reaches the CISO.

The root cause is that most agents today are ghosts. They don’t have identities. They borrow a service account, inherit a human’s OAuth token, and “promise” - in application code, in a prompt - to stay inside the lines. In a real enterprise environment, a promise in a prompt is not a policy.

My bet is that agent identity has to move from the application layer down into the platform layer.

The difference is between bolted-on vs. embedded security. Bolted-on looks like middleware in front of every tool call, politely asking the agent to behave: easy to bypass, expensive in latency, and invisible to your existing IAM. Embedded looks like a badge reader welded into a steel frame. The agent has a distinct, unforgeable identity recognized at the network and platform level, and policy is enforced at the source. If the agent reaches for a database it isn’t cleared for, the connection never opens. No middleware, no vibes.

Done right, this turns “a fleet of liabilities” into something that looks a lot more like a managed workforce: every action attributable, every permission auditable, every agent revocable with one call.

2) Agents need universal context, not scraped windows

Context management is a tax every builder is currently paying. Teams are burning a huge share of their engineering hours (and tokens) on undifferentiated plumbing - custom serialization, bespoke session stores, hand-rolled memory layers - just to keep an agent from forgetting its mission halfway through a multi-step task.

Worse, the context agents can get their hands on is usually siloed. A browser-based agent can see the open tab. A desktop wrapper can see the files a user happened to drag in. Neither of them can easily reason across the systems where the business actually lives - the CRM, the ERP, the data warehouse, the ticketing system, the transcripts, the project plans - at the same time.

Agents need universal context that integrates at the platform level. If we don’t fix this, we should be honest that the ceiling of agentic AI is “slightly better spreadsheet autocomplete,” and we should stop writing vision pieces about it.

3) Agents need to survive your laptop closing

Here’s the uncomfortable version of this: a lot of what ships today as “an agent” isn’t yet ready to deploy across a business.

I want to be precise, because the frontier has genuinely moved in the last six months. Environments like Claude Code, OpenClaw, and similar platforms are capable - persistent task state, scheduled execution, multi-agent coordination, and long-running sessions that survive disconnects are no longer aspirational. These are not toys. The question has moved on.

The question now is whether an agent can run for a week instead of an hour. Whether it can cross three handoffs, two credential rotations, and an approval gate without a human babysitting the session. Whether the work it did on Tuesday is auditable on Friday by someone who wasn’t in the room. A session that survives a dropped WebSocket is table stakes. A mission that survives a quarter is the bar enterprises actually need.

Real work doesn’t fit in a session, and most of it doesn’t fit in a day either. A procurement workflow spans weeks and a dozen handoffs. A compliance audit runs for a month. An incident investigation outlives three on-call rotations.

Most agents today hit a hard ceiling - sometimes time-based, sometimes token-based, sometimes governance-based - and when they hit it, the mission fails and a human picks up the pieces from wherever the transcript ended.

Enterprise-grade autonomy requires durable, cloud-native execution with a much higher floor than “the session stayed up.” Concretely, that means:

• State and checkpointing that survives restarts, disconnects, redeploys, and model version changes by default - not bolted on with a local Redis and a prayer.

• Context that outlives the window: long-horizon memory, summarization, and handoff between agent instances, so a multi-week task doesn’t die because a single run exhausted its tokens.

• Missions that outlive sessions: agents that stay on the job across days, handoffs, and credential rotations, with an auditable trail of what happened while you were asleep.

• First-class human-in-the-loop primitives, so the agent can pause and ask for permission to do something new instead of silently deciding it has the authority.

Persistence with guardrails. That’s the bar. Anything less and you’re building demos that happen to run for a long time.

4) Agents need platforms

The pattern I see most often in strong teams is the saddest one: brilliant engineers draining their bandwidth into stack problems that do not differentiate their product. Custom memory. Bespoke eval harnesses. Homegrown observability. Handwritten retry logic. A tracing system that almost works. None of this is the hard part of the agentic era, and none of it is what your users are paying you for.

The real value lives in domain reasoning and business logic - the judgment calls that are specific to your company, your customers, your regulatory environment. Everything underneath should be the platform you build on, not the plumbing you build.

This is why the maturation of open primitives matters right now. Open-source orchestration frameworks exist precisely so the scaffolding isn’t locked behind any single vendor’s roadmap. The model that worked for cloud compute, containers, and CI/CD - start local on open primitives, graduate to a managed platform when you’re ready to scale - is the model agent platforms need to copy.

Teams should be able to prototype on their laptop with the same building blocks they’ll run in production, and cross that boundary without a rewrite.

That’s the engineering standard that lets teams stop fighting plumbing and get back to the product.

The five-year horizon

The teams that pull ahead in the next five years will not pull ahead by being smarter at writing boilerplate. They’ll pull ahead by choosing the right agent foundation and spending their engineering hours on the problems only they can solve.

Every month spent rebuilding the common stack - identity, context, persistence, orchestration - is a month not spent on the logic that actually makes your agents worth deploying.

The agent stack has to become a solved problem. The only real question is whether you want to solve it yourself, again, or build on a foundation that was engineered for agents from the ground up.

My bet is on the latter. I think yours should be too.

Across the tools many developers including myself are already using daily - Conductor, Claude Code Web, GitHub Copilot Agent, Jules, Vibe KanBan, even cmux - the same shift keeps showing up: the control plane is becoming the primary surface, and the editor is becoming one of several instruments underneath it.

Cursor just shipped Glass - a new interface explicitly built to make “working with agents clear, intuitive, and in your control” where agent management is the primary experience and the traditional editor is something you reach for when you need to go deeper. The reaction from developers was immediate:

Now Cursor feels more like an Agent Orchestrator than an IDE. Managing agents in parallel is easier

But Glass is one data point in a much larger pattern. Terminal UIs like cmux highlight how the surfaces we’re used to are evolving to better manage agent workflows.

From editing files to steering workstreams

Historically, IDEs optimized for a tight inner loop: open files → edit → build → debug → repeat. The “death” argument is that this loop is no longer the dominant unit of productivity once agents can execute most of it autonomously.

The new loop looks like this: specify intent → delegate → observe → review diffs → merge. What makes it different from “autocomplete with a chat window” is tool-using autonomy combined with interfaces designed to make that autonomy governable.

You can see this playing out across tools already in heavy use. Claude Code Web (or Desktop) and Codex let developers hand off well-defined tasks to agents running in isolated cloud environments, with progress visible in a browser - no terminal, no local setup required.

GitHub Copilot’s Agents plan and implements multi-file changes independently, creates branches, runs tests, and surfaces a PR for review; the developer’s primary job becomes reviewing the outcome and iterating, not directing each step.

Conductor takes a different approach: a desktop app for running multiple Claude Code agents simultaneously in isolated workspaces, with live progress monitoring across all of them. And Google’s Jules handles asynchronous background tasks - you assign work, it runs, you review the result when it’s done.

What these tools share is a mental model: the agent is the unit of work, not the file. The interface worth optimizing is the one that helps you direct, monitor, and review agents - not the one that helps you type faster.

The orchestration layer taking shape

The displacement story becomes persuasive only when you look at the specific interface patterns converging across tools.

Work isolation as a primitive. Parallel agents need to not step on each other. Virtually every serious tool in this space has landed on git worktrees (or similar) as the answer. Conductor maps each agent session to its own isolated workspace. Vibe Kanban (shown above) does the same for its kanban-driven agent workflow. The pattern is near ubiquitous because the problem is real: without isolation, parallel agents produce chaos.

Planning and task state as the primary UI. Tools like Vibe Kanban have replaced “tabs and files” with “tasks and states” as the top-level mental model. You create task cards (a landing page, a backend service, an email integration), assign each to an agent and a model, and manage the whole effort like a lightweight project board - except the “team” is running autonomously. This is a project management surface that happens to have agents doing the implementation.

Background agents and async-first design. Some of the most interesting tools in this space don’t even try to keep you in the loop during execution. Cursor, Copilot and Antigravity support background agents that run without requiring your presence - you define intent, step away, and review when they’re done. Jules works similarly: assign a task, come back to a diff. The implicit promise is that your attention is too valuable to spend watching a progress bar. That’s a significant departure from the IDE’s real-time, synchronous feedback loop.

Attention management for parallel agents. When many agents run concurrently, the real bottleneck becomes knowing which one needs you right now. This is why tools like Conductor surface live progress across sessions and cmux introduced notification rings and unread badges for terminal panes. “Agent needs attention” is becoming a first-class event in the developer environment - something to route and triage, not just notice.

Agents embedded into the software lifecycle. GitHub’s Copilot coding agent is asynchronous, secured by a control layer, and powered by GitHub Actions - attached to how code actually ships (issues → PRs → CI → merge), not just how it gets written.

None of these tools claim IDEs are obsolete - many still interoperate with them. But the repeated patterns (parallel workspaces, diff-first review, task state, background execution, lifecycle integration) are precisely what “death of the IDE” proponents mean when they talk about a center-of-gravity shift.

Why developers still reach for an IDE

The best critique of “the IDE is dead” is that the IDE still compresses several genuinely hard problems into a high-fidelity feedback loop: precise navigation, local reasoning, interactive debugging, and the ability to understand a system by directly manipulating it.

Even the most ambitious orchestration tools keep a manual-edit escape hatch. For example, reviewing diffs in-thread, commenting on changes, and then opening the result in your editor for manual adjustments. That’s an acknowledgment that human intervention is part of the intended workflow.

Agent tooling itself highlights where the limits still are. Multi-file refactorings in large repositories remain among the toughest challenges for software engineering agents. These are exactly the situations where interactive code navigation and human judgment still matter most - where you need to hold a mental model of the system that the agent can’t fully reconstruct from context alone.

The failure mode that keeps developers anchored to IDE-level inspection is agents being almost right. When something is 90% correct and subtly broken, the cost of finding the issue often exceeds what it would have taken to write it yourself. For high-stakes changes, the IDE remains the best instrument for that kind of deep, precise inspection.

The new costs: review fatigue and governance overhead

If development becomes “run many agents in parallel” the workflow inherits problems that look less like text editing and more like distributed systems management - observability, permissions, isolation, and governance.

Agent workflows invert the labor. Instead of writing, you’re reviewing. That sounds like an improvement until you’re staring at twelve diffs from twelve parallel agents at the end of the day. Review fatigue is real, and it’s one of the reasons the most thoughtful tools in this space focus on attention routing, structured plans, and review-first gates rather than pushing for full autonomy by default.

The security surface also expands as agents gain access to more tools, repos, and external systems. As agents can browse the web, query databases, write to filesystems, and trigger deploys, what they’re allowed to do becomes as important as what they’re capable of doing.

On observability and control, IDE-integrated agent modes are already pushing toward explicit tool logs and approval gates. The governance question isn’t optional once agents act asynchronously and touch CI pipelines.

What survives: the IDE, the control plane, or both

A clear reading of the landscape is that “death of the IDE” is directionally right about the center of gravity, but wrong as a literal forecast.

The strongest version of the claim is this: the IDE stops being the primary workspace and becomes one of several subordinate instruments - used for targeted inspection, debugging, and final edits - while planning, orchestration, review, and agent management move into dashboards, issue trackers, observability terminals, and cloud control planes.

The “bigger IDE” framing is equally well-supported. The new “IDE” is a system that provides multi-agent orchestration, isolated workspaces, permissions and audit logs, diff-first review, reliable tool connectivity, and attention routing. The file editor is still there. It’s just no longer the front door.

The IDE isn’t dying. It’s being de-centered. The work is moving outward - into orchestration surfaces where humans define intent, delegate to parallel agent runtimes, and spend more time supervising, reviewing, and governing than typing.

The IDE remains critical for correctness, comprehension, and the hard problems agents still struggle with. But its no longer the only place where programming happens - and for a growing number, it’s no longer the first place they go.

That made me realize I’d left a lot on the table. The first list skewed toward individual craft - how to write better code, how to think about your career. But some of the hardest lessons I’ve learned aren’t about how you work. They’re about how teams work: how decisions actually get made, where coordination breaks down, what separates the groups that ship from the ones that spin.

These lessons pick up where the first left off. They’re less about being a better individual engineer and more about the systems around the engineering.

1. The best engineers pick the right problems to solve.

Every yes is an implicit no to something else.

I’ve watched talented engineers burn out because they said yes to everything - every bug, every feature request, every “quick favor.” Their calendar filled up with other people’s priorities, and their own roadmap became a graveyard of half-finished ideas.

Sometimes it’s just because they truly do care so much about the product. Protect your bandwidth from “nice to have” the same way you protect production from outages. The skill is doing the right things and letting the wrong things stay undone.

The engineers who create disproportionate impact aren’t necessarily faster or smarter. They’re more ruthless about what deserves their attention. They’ve learned that the opportunity cost of working on the wrong thing is working on the wrong thing.

2. If you can’t say what decision you’re asking for, you’re not ready for the meeting.

Most meetings fail not because they’re unnecessary, but because they’re disguised journaling. I’ve sat through hundreds of hours where smart people talked around a problem without ever naming what they needed. The meeting ends with vibes and no owner.

I learned to start with the ask: approve, choose, unblock, or inform.

Just those four words changed how I prepare for every meeting. If I can’t pick one, I’m not ready to take anyone’s time. And when I’m on the receiving end, I’ve started asking “what decision do you need from me?” within the first two minutes. It sounds blunt, but people are usually relieved - they often didn’t realize they hadn’t defined it themselves.

The hidden cost of vague meetings isn’t just the hour you lose. It’s the week of drift that follows while everyone waits for clarity that never came.

3. “We should” is not a plan. “On Tuesday, I will” is a plan.

The difference between motion and progress is specificity.

Teams drown in intentions. I’ve watched roadmaps fill up with “we should improve the onboarding flow” and “we should reduce latency” and “we should document the API.” Months later, the same items are still there, gathering dust and guilt. You might think that’s a solved problem now that we have agentic engineering, but not quite.

Convert talk into the smallest next action someone can actually do, then put a name and a date on it. Not “we should improve onboarding” but “On Tuesday, Sarah will run three user sessions and document the top friction points.”

This is about respecting that humans need traction to make progress. Vague intentions create anxiety. Specific commitments create momentum. The plan doesn’t have to be perfect - it just has to be concrete enough that someone can actually start.

4. Slow code is sometimes a symptom. Slow decisions are always a problem.

Speed is about removing the friction that makes smart people hesitate. “Bias towards action” when you can.

When a project drags, the instinct is to blame velocity: people aren’t working hard enough, the codebase is messy, there aren’t enough engineers. But in my experience, slow code is often a symptom. Slow decisions are the disease.

If decisions routinely take weeks or months, look deeper. Missing context means people can’t evaluate tradeoffs. Unclear ownership means everyone’s waiting for someone else to decide. Fear of accountability means people hedge instead of commit.

The fastest engineering team I ever worked with wasn’t the one with the best programmers. It was the one where decisions happened in hours instead of weeks because the authority was clear, the context was shared, and being wrong wasn’t a career risk.

5. Reliability is a product feature. Treat it like one.

Users don’t praise reliability but they do notice its absence.

This creates a dangerous dynamic: reliability work is invisible until it fails, which means it’s perpetually under-resourced compared to shiny new features.

Error budgets are one way to make the tradeoff explicit. If your service has an SLO of 99.9% uptime, you have a “budget” of 0.1% downtime to spend on innovation. Burn through it, and you focus on reliability until you’ve earned it back. This is a framework for having honest conversations about risk.

The teams that maintain both velocity and reliability don’t do it through heroics. They do it by treating reliability as a first-class product feature with its own roadmap, its own metrics, and its own advocates.

You wouldn’t ship a feature without product review. Don’t ship a system without some kind of reliability discussion.

6. You can’t “communication” your way out of a bad interface between teams.

Team interaction modes exist for a reason: collaboration (working closely together), service (clear API and SLAs), or facilitation (one team helping another build capability).

Most cross-team pain isn’t about effort or good intentions. It’s about unclear boundaries and messy contracts. I’ve watched teams “improve communication” by adding more meetings, more Slack channels, more syncs - and it doesn’t make things better.

The problem isn’t that people aren’t talking. It’s that the interface between teams is undefined. Who owns what? What’s the contract? What can team A depend on team B for, and vice versa?

Choose deliberately, and you’ll need fewer meetings to make things work. Try to paper over a bad interface with communication, and you’ll burn out your most collaborative people while the underlying dysfunction remains.

7. The best escalation comes with a proposal.

“Here’s the problem” is half the job. I used to think my role was to identify issues and bring them to leadership. That’s necessary but insufficient.

“Here are two options, the tradeoffs, and what I recommend” is how you get unblocked and earn trust. It shows you’ve done the thinking. It gives decision-makers something super specific to react to instead of an open-ended problem to solve.

It makes their job easier, which makes them more likely to give you what you need.

The difference between “I need help” and “I need you to choose between A and B, and here’s why I lean toward B” is the difference between being a problem-raiser and being a problem-solver.

Both identify issues. Only one earns increasing trust and autonomy.

8. Avoid hero culture. Build systems that don’t require heroes.

The hero is burned out, undocumented, and a single point of failure.

If one person saving the day is a recurring pattern, that’s a failure mode rather than a badge of honor. I’ve seen teams celebrate their heroes while ignoring the dysfunction that made heroism necessary.

When they leave - and they always leave eventually - the team discovers that no one else really knows how things work. The celebration of heroism masks a systemic problem: the path for “normal humans on a normal day” doesn’t work.

Make the normal path the default. Document the system. Spread the knowledge. Design for the average Tuesday, not the exceptional crisis. Heroes should be unnecessary, and if they’re necessary, you should be working to make them unnecessary.

9. Make observability part of the feature.

A feature without telemetry is a liability in disguise.

If you ship a feature without knowing how it behaves in production, you shipped uncertainty.

I’ve watched teams celebrate launches only to discover weeks later that their feature was silently failing for 20% of users. They had no logs, no metrics, no dashboards but a gap where understanding should be. This can cause all kinds of pain if you want to fix it, including unshipping just to properly A/B test with observability in place.

Logs, traces, dashboards, and alerts aren’t “ops work.” They’re how you learn. They’re how you know whether the thing you built actually works for real people doing real things in real conditions.

The best engineers I know treat observability as part of the definition of done. Not “I wrote the code” but “I wrote the code and I can see it working.”

10. Small PRs are kindness. Especially if the PR is AI generated.

Small changes are easier to review, easier to reason about, and easier to revert.

I used to write large pull requests. I liked the idea of a complete feature being reviewable at once. I was optimizing for my convenience at the expense of my reviewers’ sanity. Smaller PRs are often better for everyone.

They ship faster because they don’t sit in a review queue while someone tries to find an hour to understand your thousand-line diff. If you want teammates to trust your pace, make your work reviewable.

The hidden benefit is that small PRs force you to think in increments. Instead of one monolithic change, you build up capability piece by piece. Each piece gets feedback. Each piece can be rolled back independently. It’s slower per-PR but faster to actual production.

11. When you add a team, you add edges, not just nodes.

Coordination cost grows faster than headcount.

This is why “just throw more people at the problem” often fails, and why adding heads late in a project can make it later. Every new person adds communication overhead with everyone they need to coordinate with. The graph gets denser, not just larger.

I’ve seen managers genuinely puzzled when a team doubled in size but output barely changed. The answer is always the same: the new edges ate the new capacity. More people meant more alignment meetings, more context-sharing, more waiting for decisions that now required more stakeholders.

The solution isn’t to stop hiring. It’s to be intentional about reducing edges. Clear ownership. Autonomous teams with minimal dependencies. Interfaces that let people work in parallel instead of in lockstep. The best organizations aren’t the ones with the most people - they’re the ones with the most leverage per person.

12. The migration is never just a migration

Every migration is a negotiation between the system you have, the system you want, and the people who didn’t ask for either.

I’ve seen migrations estimated at one quarter stretch to years. Not because the technical work was wrong, but because nobody accounted for the human work: convincing teams to prioritize your migration over their roadmap, supporting the long tail of edge cases nobody knew existed, and maintaining two systems in parallel while the old one refuses to die.

The technical plan is the easy part. The hard part is designing for coexistence. You will run old and new simultaneously for longer than you think. You will discover that the “legacy” system encodes decisions nobody documented and workflows nobody remembers designing but everyone depends on. You will need a adoption strategy that doesn’t require every team to drop what they’re doing at once.

The migrations that actually finish share three traits: a sponsor who stays engaged past the kickoff, a team that really owns the migration instead of treating it as a side quest, and a clear deprecation date that people believe is real. Without all three, you get a migration that’s perpetually “almost done” - which is worse than not starting, because now you’re paying the cost of two systems indefinitely.

If you’re not willing to fund the finish, don’t start the migration.

13. AI makes drafts cheap. Taste becomes expensive.

Everyone can generate code now. The barrier to producing code, content, designs - it’s largely collapsing. AI will write you ten versions of anything in the time it used to take to write one.

The differentiator is choosing: what to build, what to delete, what to simplify, what not to ship, and what “good” looks like. Taste - the ability to distinguish between options and pick the right one - becomes the scarce resource.

Use AI to explore options fast, then apply judgment ruthlessly. The engineers who thrive in this environment won’t be the ones who generate the most. They’ll be the ones who curate the best.

Production is cheap. Editing is expensive. Selection is everything.

14. Trust is a latency optimization for teams.

This is the highest-leverage thing you can build. Not a system but credibility.

When people trust you, they don’t need five meetings to approve a decision. They assume competence, good intent, and follow-through. Decisions that would take weeks in a low-trust environment take hours in a high-trust one.

Every time you deliver on a promise, every time you’re honest about a mistake, every time you make someone else’s life easier, you’re depositing into an account that will pay dividends for years.

I’ve watched engineers with modest technical skills accomplish enormous things because everyone trusted them. I’ve watched brilliant engineers accomplish little because nobody would take their calls.

The code doesn’t matter if you can’t get anyone to ship it with you.

A final thought

The first time around, I said these lessons come down to staying curious, staying humble, and remembering that the work is about people. I still believe that.

But if this second list has a through-line, it’s something more specific: the work is about making it easier for normal people to do extraordinary things on a normal day. A career in engineering gives you plenty of time to learn these things the hard way and I’ve certainly learned a lot during my time at Google so far.

I hope a few of them save you a scar or two. And if they do, share what you’ve figured out with someone earlier in the journey.

That’s how the good lessons travel.

“I rapidly went from about 80% manual+autocomplete coding and 20% agents to 80% agent coding and 20% edits+touchups. I really am mostly programming in English now.”

The inversion happened over a few weeks in late 2025. While this may apply to new (greenfield) or personal projects more than existing or legacy apps, I imagine how far AI takes you is still further than a year ago. You can thank models, specs, skills, MCPs and our workflows improving.

Boris Cherney, creator of Claude Code, has recently echoed similar sentiments:

“Pretty much 100% of our code is written by Claude Code + Opus 4.5. For me personally it has been 100% for two+ months now, I don’t even make small edits by hand. I shipped 22 PRs yesterday and 27 the day before, each one 100% written by Claude. I think most of the industry will see similar stats in the coming months - it will take more time for some vs others.”

Some time ago I wrote about “the 70% problem” - where AI coding took you to 70% completion, then leave the final 30% last mile for humans. That framing may now be evolving. The percentage may shift to 80% or higher for certain kinds of projects, but the nature of the problem changed more dramatically than the numbers suggest.

Armin Ronacher’s poll of 5,000 developers compliments this story: 44% now write less than 10% of their code manually. Another 26% are in the 10-50% range. We’ve crossed a threshold. But here’s what the triumphalist narrative misses: the problems didn’t disappear, they shifted. And some got worse.

I want to caveat: I’ve definitely felt the shift to 80%+ agent coding on new side-projects, however, this is very different in large or existing apps, especially where teams are involved. Expectations differ, but this is a taste of where we’re headed.

The mistakes changed

AI errors evolved from syntax bugs to conceptual failures - the kind a sloppy, hasty junior may make under time pressure.

Karpathy catalogs what still breaks:

“The models make wrong assumptions on your behalf and run with them without checking. They don’t manage confusion, don’t seek clarifications, don’t surface inconsistencies, don’t present tradeoffs, don’t push back when they should. They’re still a little too sycophantic.”

Assumption propagation: The model misunderstands something early and builds an entire feature on faulty premises. You don’t notice until you’re five PRs deep and the architecture is cemented. This is kind of two-steps-back pattern.

Abstraction bloat: Given free rein, agents can overcomplicate relentlessly. They’ll scaffold 1,000 lines where 100 would suffice, creating elaborate class hierarchies where a function would do. You have to actively push back: “Couldn’t you just...?” The response is always “Of course!” followed by immediate simplification. They’re optimizing for looking comprehensive, not for maintainability.

Dead code accumulation: They often don’t clean up after themselves. Old implementations linger. Comments get removed as side effects. Code they don’t fully understand gets altered anyway because it was adjacent to the task.

Sycophantic agreement: They don’t always push back. No “Are you sure?” or “Have you considered...?” Just enthusiastic execution of whatever you described, even if your description was incomplete or contradictory.

It’s possible to mitigate some of this via Skills if you know what to watch for.

These otherwise persist despite system prompts, despite CLAUDE.md instructions, despite plan mode. They’re not bugs to be fixed - they’re sometimes inherent to how these systems work.

Agents optimize for coherent output, not for questioning your premises.

I've watched this happen on my own teams - code that looks right in review but breaks three commits later when someone touches an adjacent system.

If you’re data minded, recent survey data suggests “verification bottleneck” has emerged: only 48% of developers consistently check AI-assisted code before committing it, even though 38% find that reviewing AI-generated logic actually requires more effort than reviewing human-written code. We’re generating correct code faster, but may be accumulating technical debt even faster.

Comprehension debt: a hidden cost we don’t track

Generation (writing code) and discrimination (reading code) are different cognitive capabilities. You can review code competently even after your ability to write it from scratch has atrophied. But there’s a threshold where “review” becomes “rubber stamping.”

Jeremy Twei coined the perfect term for this: comprehension debt. It’s certainly tempting to just move on when the LLM one-shotted something that seems to work. This is the insidious part. The agent doesn’t get tired. It will sprint through implementation after implementation with unwavering confidence. The code looks plausible. The tests pass (or seem to). You’re under pressure to ship. You move on.

Over time, you may understand less of your own codebase.

I caught myself doing this last week. Claude implemented a feature I’d been putting off for days. The tests passed. I skimmed it, nodded, merged. Three days later I couldn’t explain how it worked.

Yoko Li captured the addiction loop perfectly:

“The agent implements an amazing feature and got maybe 10% of the thing wrong, and you’re like ‘hey I can fix this if I just prompt it for 5 more mins.’ And that was 5 hrs ago.”

You’re always almost there. The final 10% feels tantalizingly close. Just one more prompt. Just one more iteration. The psychological hook is real.

Someone else put it differently:

“I spend most of my time babysitting agents. The AGI vibes are real, but so is the micromanagement tax. You’re not coding anymore, you’re supervising. Watching. Redirecting. It’s a different kind of exhausting.”

The dangerous part: it’s trivially easy to review code you can no longer write from scratch. If your ability to “read” doesn’t scale with the agent’s ability to “output,” you’re not engineering anymore. You’re hoping.

The productivity paradox: More code, same throughput

Individual output surged 98% in high-adoption teams, but PR review time increased anywhere as high as 91%.

The data from Faros AI and Google’s DORA report are interesting:

• Teams with high AI adoption merged 98% more PRs

• Those same teams saw review times balloon 91%

• PR size increased 154% on average

• Code review became the new bottleneck

Atlassian’s 2025 survey found the paradox in stark terms: 99% of AI-using developers reported saving 10+ hours per week, yet most reported no decrease in overall workload. The time saved writing code was consumed by organizational friction - more context switching, more coordination overhead, managing the higher volume of changes.

We got faster cars, but the roads got more congested.

We're producing more code but spending more time reviewing it. The bottleneck just moved. When you make a resource cheaper (in this case, code generation), consumption increases faster than efficiency improves, and total resource use goes up.

We’re not writing less code. We’re writing vastly more code, and someone still has to understand much of it. There are of course groups of developers who feel this should no longer be the case if AI can do that.

Where the 80/20 split actually works

The 80% threshold is most accessible in greenfield contexts where you control the entire stack and comprehension debt stays manageable through small team size.

This actually works in a few contexts.

• Personal projects where you control everything

• MVPs where “good enough” is actually good enough

• Startups in greenfield territory without legacy constraints

• Teams small enough that comprehension debt stays manageable

In these environments, the agent’s weaknesses matter less. You can scaffold rapidly, refactor aggressively, throw away code without political friction. The pace of iteration outweighs occasional misdirection.

In mature codebases with complex invariants, the calculus inverts. The agent doesn’t know what it doesn’t know. It can’t intuit the unwritten rules. Its confidence scales inversely with context understanding.

Someone pointed out the obvious thing I was tiptoeing around: the first 90% might be easy, but the last 10% can take a long time. 90% accuracy is fine for non-mission-critical stuff. For the parts that actually matter, it's nowhere close. Self-driving cars work great until they don't, and that's why L2 is everywhere but L4 is still mostly vaporware.

For non-engineers, the wall is lower but still real. Tools like AI Studio, v0 and Bolt can turn sketches into working prototypes instantly. But hardening that prototype for production - handling real user data at scale, ensuring security and compliance - still requires engineering fundamentals. AI gets you 80% to an MVP; the last 20% requires patience, learning deeply or hiring engineers.

Two different populations

We’re not seeing a smooth curve of adoption - we’re seeing a split between those who’ve crossed the threshold and everyone else. The gap between early adopters and the rest is widening, not closing.

Armin’s poll revealed what raw adoption numbers obscure: 44% of developers still write over 90% of their code manually. We have a bimodal distribution, not a bell curve. On one side: people like Karpathy and the Claude Code team, shipping dozens of PRs daily with 100% AI-written code, iterating faster than ever before. On the other: the vast majority, incrementally adopting copilot-style tools but not fundamentally changing their workflow.

The age split may be visible in discourse too. Younger developers seem more willing to adapt workflow radically. Older developers are more skeptical - not because they can't use the tools, but because they've seen enough cycles to know the difference between a temporary productivity boost and a sustainable practice. Both might be right.

Stack Overflow’s 2025 survey showed only 16% reported “great” productivity improvements. Half saw modest gains. The top frustrations: “AI solutions that are almost right, but not quite” (66%) and “debugging AI code takes longer than writing it myself” (45%).

The engineers who appear to be thriving in 2026 aren’t just using better tools. They’ve reconceptualized their role from implementer to orchestrator. They’ve learned to think declaratively rather than imperatively. They’ve accepted that their job is now architectural oversight and quality control, not line-by-line coding.

Those struggling are trying to use AI as a faster typewriter. They haven’t adapted their workflow. They’re fighting the agent’s approach instead of redirecting its goals. They haven’t invested in learning to prompt effectively which is now as critical as writing good documentation or design specs ever was.

There's an uncomfortable truth here: orchestrating agents feels a lot like management. Delegating tasks. Reviewing output. Redirecting when things go sideways. If you became an engineer because you didn't want to be a manager, this shift might feel like a betrayal. The role changed underneath you.

The gap seems to be widening. The people who’ve figured out how to work with these tools are shipping stuff I can barely keep up with. Everyone else is... still figuring it out.

This split may make some uncomfortable. I’ve always said I’m a builder, but I also enjoyed programming. The idea that these are now diverging paths - that you have to pick one - feels reductive. Like we’re forcing a binary on something more complicated. Someone in the comments said it perfectly: both viewpoints are valid, just different wiring. Neither is wrong.

From imperative to declarative: The real leverage

Don’t tell the AI what to do - give it success criteria and watch it loop. The magic isn’t in the agent writing code, it’s in the agent iterating until it satisfies conditions you specify.

Karpathy’s observation about leverage cuts to the core:

“LLMs are exceptionally good at looping until they meet specific goals and this is where most of the ‘feel the AGI’ magic is to be found.”

The shift from imperative to declarative development:

Old model (imperative): “Write a function that takes X and returns Y. Use this library. Handle these edge cases. Make sure to...”

New model (declarative): “Here are the requirements. Here are the tests that must pass. Here’s the success criteria. Figure out how.”

This works because agents never get demoralized. They’ll try approaches you wouldn’t have patience for. They iterate relentlessly. If you specify the destination clearly, they’ll navigate there - even if it takes 30 failed attempts.

The patterns that work:

• Write tests first, let the agent iterate until they pass

• Hook it up to a browser via MCP, let it verify behavior visually

• Implement the naive correct version, then optimize while preserving correctness

• Define the API contract, let it implement to spec

But this only works if your success criteria are actually correct. Garbage in, garbage out scales with capability.

The developers succeeding with this approach spend 70% of their time on problem definition and verification strategy, 30% on execution. The ratios inverted from traditional development, but the total time decreased dramatically.

The slopacolypse question

When anyone can generate thousands of lines of code in minutes, the ability to say ‘we don’t need this’ becomes more valuable.

Karpathy warned:

“I am bracing for 2026 as the year of the slopacolypse across all of github, substack, arxiv, X/instagram, and generally all digital media.”

The concern is straightforward: when anyone can generate arbitrarily large volumes of plausible-looking code, content, papers, or posts, how do we maintain signal-to-noise ratio?

Boris Cherny offers a counterpoint: “My bet is that there will be no slopcopolypse because the model will become better at writing less sloppy code and at fixing existing code issues. In the meantime, what helps is having the model code review its code using a fresh context window.”

Both can be true simultaneously. The capability for slop exists at unprecedented scale. The tooling to prevent it is emerging. The question is which scales faster.

The slopacolypse will be driven by people who mistake velocity for productivity. Agents are marathon runners with no sense of direction unless you give it to them. They will sprint ten miles into a brick wall if you don’t audit the “code actions” where necessary.

The teams I’ve seen handle this well tend to do a few things:

• Fresh-context code reviews help, though it feels weird asking the same model to critique its own code. It works, though - give it a clean slate and it catches its own mistakes.

• Automated verification at every step (CI/CD, linters, type checkers, tests as guardrails)

• Deliberate constraints on agent autonomy (bounded tasks, clear success criteria)

• High emphasis on human-in-the-loop at architectural decision points

The code quality problems Karpathy describes - overcomplication, abstraction bloat, dead code - these improve as models improve. But they won’t disappear. They’re emergent from how these systems approach problems.

What actually works: practical patterns

The future belongs to those who can maintain a coherent mental model of the macro while agents handle tactical drudgery of the micro.

After watching teams adapt over the past year, effective patterns have crystallized:

1. Agent-first drafts with tight iteration loops

Don’t use AI for one-off suggestions. Generate entire first drafts, then refine. The Claude Code team practice: have the model review its own code with a fresh context window. This catches issues before human review.

2. Declarative communication

Spend 70% of effort on problem definition, 30% on execution. Write comprehensive specs, define success criteria, provide test cases up front. Guide the agent’s goals, not its methods.

3. Automated verification

If you repeatedly fix the same class of mistake, write a test or lint rule preemptively. Make the agent explain its code and flag potential problems before you review.

4. Deliberate learning vs. just focusing so much on production

Use AI as a learning tool, not a crutch (you’ve heard this a few times now). When the agent writes something you don’t understand, that’s a signal to dig deeper. Treat AI-generated code like code from a mentor - review it to learn, not just to ship.

5. Architectural hygiene

More modularization, clearer API boundaries. Well-documented style guides fed into prompts. High-level architecture descriptions provided before coding begins. The planning phase expanded; the coding phase compressed; the review phase focused on design rather than syntax.

The developers who thrive won’t be those who generate the most code. They’ll be those who know which code to generate, when to question the output, and how to maintain comprehension even as their hands leave the keyboard.

The uncomfortable truth about skill development

If your ability to “read” doesn’t scale at the same rate as the agent’s ability to “output,” you aren’t engineering anymore. You’re rubber stamping.

“It’s been like the boiling frog for me. Started by copy-pasting more into ChatGPT. Then more in-IDE prompting. Then agent tools. Suddenly I barely hand code anymore. The transition was so gradual I didn’t notice until I was already there” [HN]

There’s early evidence of skill atrophy in heavy AI users. Junior developers who rely on AI for everything report feeling less confident in problem-solving abilities over time. It’s the Google effect applied to coding - when you outsource constantly, your brain stops retaining.

I don’t know what the solution is, but I’ve been trying a few things:

• Use TDD: write tests (or think through test cases) before letting AI implement

• Pair with seniors: discuss AI suggestions in real-time to learn the decision-making process

• Ask for explanations: have the AI justify its approach, not just generate solutions

• Alternate: write some features manually to maintain muscle memory

The risk is real: it’s dangerously easy to review code you can no longer write from scratch. When that happens, you’ve become dependent on the tool in a way that limits your growth.

The engineers who will thrive long-term are those who use AI to accelerate gaining experience, not to bypass it entirely. They maintain their fundamentals while leveraging AI to explore more territory faster.

Where this leaves us

The shift from 70% to 80% isn’t about percentages - it’s about the gap between prototype and production-ready software. That gap is narrowing, but it hasn’t closed.

Karpathy asks the right questions:

“What happens to the ‘10X engineer’ - the ratio of productivity between the mean and the max engineer? It’s quite possible that this grows a lot. Armed with LLMs, do generalists increasingly outperform specialists?”

These questions will define the next few years.

One thing is certain: AI wrote 80% of code for early adopters in late 2025. Even if your percentage is much lower, it’s likely higher than a year ago. This places disproportionate emphasis on the human’s role: owning outcomes, maintaining quality bars, ensuring tests actually validate behavior.

The danger isn’t that the agent fails. I think it’s that it succeeds so confidently in the wrong direction that you stop checking the compass.

DORA’s 2025 report crystallized the reality: AI is an amplifier of your development practices. Good processes get better (high-performing teams saw 55-70% faster delivery). Bad processes get worse (accumulating debt at unprecedented speed). There is no silver bullet.

Karpathy’s final observation resonates most:

“I didn’t anticipate that with agents programming feels more fun because a lot of the fill in the blanks drudgery is removed and what remains is the creative part. I also feel less blocked/stuck and I experience a lot more courage because there’s almost always a way to work hand in hand with it to make some positive progress.”

He also notes: “LLM coding will split up engineers based on those who primarily liked coding and those who primarily liked building.”

That’s probably the most insightful prediction about where this is headed.

If you liked the act of writing code itself - the craft of it, the meditation of it - this transition might feel like loss. If you liked building things and code was the necessary means, this feels like liberation.

Neither response is wrong. But the tooling is optimizing for the latter.

For the skeptics (you’re right to be skeptical)

The productivity claims are often overhyped. AI still makes mistakes a competent junior wouldn’t. Comprehension debt is real and poorly understood. The slopacolypse risk is genuine.

But the shift is real. When Karpathy admits he barely writes code directly anymore, when the Claude Code team ships 20+ PRs daily with 100% AI-written code, we’re past the point of dismissing this as hype.

As software engineers, our identity was never “the person who can write code” - it was “the person who can solve problems with software.”

AI isn’t replacing engineers. It’s amplifying them - for better and for worse.

My advice: embrace the tools, but own the outcome. Use AI to accelerate learning, not skip it. Focus on fundamentals that matter more than ever: robust architecture, clean code, thorough tests, thoughtful UX. These remain as important as ever - maybe more so, since implementation is no longer the bottleneck.

I don’t know where this goes. Karpathy’s probably right that it’ll split people between those who liked coding and those who liked building.

We’re all figuring this out in public, one PR at a time.

“I’ve heard a lot about writing good specs for AI agents, but haven’t found a solid framework yet. I could write a spec that rivals an RFC, but at some point the context is too large and the model breaks down.”

Many developers share this frustration. Simply throwing a massive spec at an AI agent doesn’t work - context window limits and the model’s “attention budget” get in the way. The key is to write smart specs: documents that guide the agent clearly, stay within practical context sizes, and evolve with the project. This guide distills best practices from my use of coding agents including Claude Code and Gemini CLI into a framework for spec-writing that keeps your AI agents focused and productive.

We’ll cover five principles for great AI agent specs, each starting with a bolded takeaway.

1. Start with a high-level vision and let the AI draft the details

Kick off your project with a concise high-level spec, then have the AI expand it into a detailed plan.

Instead of over-engineering upfront, begin with a clear goal statement and a few core requirements. Treat this as a “product brief” and let the agent generate a more elaborate spec from it. This leverages the AI’s strength in elaboration while you maintain control of the direction. This works well unless you already feel you have very specific technical requirements that must be met from the start.

Why this works: LLM-based agents excel at fleshing out details when given a solid high-level directive, but they need a clear mission to avoid drifting off course. By providing a short outline or objective description and asking the AI to produce a full specification (e.g. a spec.md), you create a persistent reference for the agent. Planning in advance matters even more with an agent - you can iterate on the plan first, then hand it off to the agent to write the code. The spec becomes the first artifact you and the AI build together.

Practical approach: Start a new coding session by prompting:

“You are an AI software engineer. Draft a detailed specification for [project X] covering objectives, features, constraints, and a step-by-step plan.”

Keep your initial prompt high-level - e.g. “Build a web app where users can track tasks (to-do list), with user accounts, a database, and a simple UI”.

The agent might respond with a structured draft spec: an overview, feature list, tech stack suggestions, data model, and so on. This spec then becomes the “source of truth” that both you and the agent can refer back to. GitHub’s AI team promotes spec-driven development where “specs become the shared source of truth… living, executable artifacts that evolve with the project”. Before writing any code, review and refine the AI’s spec. Make sure it aligns with your vision and correct any hallucinations or off-target details.

Use Plan Mode to enforce planning-first: Tools like Claude Code offer a Plan Mode that restricts the agent to read-only operations - it can analyze your codebase and create detailed plans but won’t write any code until you’re ready. This is ideal for the planning phase: start in Plan Mode (Shift+Tab in Claude Code), describe what you want to build, and let the agent draft a spec while exploring your existing code. Ask it to clarify ambiguities by questioning you about the plan. Have it review the plan for architecture, best practices, security risks, and testing strategy. The goal is to refine the plan until there’s no room for misinterpretation. Only then do you exit Plan Mode and let the agent execute. This workflow prevents the common trap of jumping straight into code generation before the spec is solid.

Use the spec as context: Once approved, save this spec (e.g. as SPEC.md) and feed relevant sections into the agent as needed. Many developers using a strong model do exactly this - the spec file persists between sessions, anchoring the AI whenever work resumes on the project. This mitigates the forgetfulness that can happen when the conversation history gets too long or when you have to restart an agent. It’s akin to how one would use a Product Requirements Document (PRD) in a team: a reference that everyone (human or AI) can consult to stay on track. Experienced folks often “write good documentation first and the model may be able to build the matching implementation from that input alone” as one engineer observed. The spec is that documentation.

Keep it goal-oriented: A high-level spec for an AI agent should focus on what and why, more than the nitty-gritty how (at least initially). Think of it like the user story and acceptance criteria: Who is the user? What do they need? What does success look like? (e.g. “User can add, edit, complete tasks; data is saved persistently; the app is responsive and secure”). This keeps the AI’s detailed spec grounded in user needs and outcome, not just technical to-dos. As the GitHub Spec Kit docs put it, provide a high-level description of what you’re building and why, and let the coding agent generate a detailed specification focusing on user experience and success criteria. Starting with this big-picture vision prevents the agent from losing sight of the forest for the trees when it later gets into coding.

2. Structure the spec like a professional PRD (or SRS)

Treat your AI spec as a structured document (PRD) with clear sections, not a loose pile of notes.

Many developers treat specs for agents much like traditional Product Requirement Documents (PRDs) or System Design docs - comprehensive, well-organized, and easy for a “literal-minded” AI to parse. This formal approach gives the agent a blueprint to follow and reduces ambiguity.

The six core areas: GitHub’s analysis of over 2,500 agent configuration files revealed a clear pattern: the most effective specs cover six areas. Use this as a checklist for completeness:

1. Commands: Put executable commands early - not just tool names, but full commands with flags: npm test, pytest -v, npm run build. The agent will reference these constantly.

2. Testing: How to run tests, what framework you use, where test files live, and what coverage expectations exist.

3. Project structure: Where source code lives, where tests go, where docs belong. Be explicit: “src/ for application code, tests/ for unit tests, docs/ for documentation.”

4. Code style: One real code snippet showing your style beats three paragraphs describing it. Include naming conventions, formatting rules, and examples of good output.

5. Git workflow: Branch naming, commit message format, PR requirements. The agent can follow these if you spell them out.

6. Boundaries: What the agent should never touch - secrets, vendor directories, production configs, specific folders. “Never commit secrets” was the single most common helpful constraint in the GitHub study.

Be specific about your stack: Say “React 18 with TypeScript, Vite, and Tailwind CSS” not “React project.” Include versions and key dependencies. Vague specs produce vague code.

Use a consistent format: Clarity is king. Many devs use Markdown headings or even XML-like tags in the spec to delineate sections, because AI models handle well-structured text better than free-form prose. For example, you might structure the spec as:

# Project Spec: My team's tasks app

## Objective
- Build a web app for small teams to manage tasks...

## Tech Stack
- React 18+, TypeScript, Vite, Tailwind CSS
- Node.js/Express backend, PostgreSQL, Prisma ORM

## Commands
- Build: `npm run build` (compiles TypeScript, outputs to dist/)
- Test: `npm test` (runs Jest, must pass before commits)
- Lint: `npm run lint --fix` (auto-fixes ESLint errors)

## Project Structure
- `src/` – Application source code
- `tests/` – Unit and integration tests
- `docs/` – Documentation

## Boundaries
- ✅ Always: Run tests before commits, follow naming conventions
- ⚠️ Ask first: Database schema changes, adding dependencies
- 🚫 Never: Commit secrets, edit node_modules/, modify CI config

This level of organization not only helps you think clearly, it helps the AI find information. Anthropic engineers recommend organizing prompts into distinct sections (like <background>, <instructions>, <tools>, <output_format> etc.) for exactly this reason - it gives the model strong cues about which info is which. And remember, “minimal does not necessarily mean short” - don’t shy away from detail in the spec if it matters, but keep it focused.

Integrate specs into your toolchain: Treat specs as “executable artifacts” tied to version control and CI/CD. The GitHub Spec Kit uses a four-phase, gated workflow that makes your specification the center of your engineering process. Instead of writing a spec and setting it aside, the spec drives the implementation, checklists, and task breakdowns. Your primary role is to steer; the coding agent does the bulk of the writing. Each phase has a specific job, and you don’t move to the next one until the current task is fully validated:

1. Specify: You provide a high-level description of what you’re building and why, and the coding agent generates a detailed specification. This isn’t about technical stacks or app design - it’s about user journeys, experiences, and what success looks like. Who will use this? What problem does it solve? How will they interact with it? Think of it as mapping the user experience you want to create, and letting the coding agent flesh out the details. This becomes a living artifact that evolves as you learn more.

2. Plan: Now you get technical. You provide your desired stack, architecture, and constraints, and the coding agent generates a comprehensive technical plan. If your company standardizes on certain technologies, this is where you say so. If you’re integrating with legacy systems or have compliance requirements, all of that goes here. You can ask for multiple plan variations to compare approaches. If you make internal docs available, the agent can integrate your architectural patterns directly into the plan.

3. Tasks: The coding agent takes the spec and plan and breaks them into actual work - small, reviewable chunks that each solve a specific piece of the puzzle. Each task should be something you can implement and test in isolation, almost like test-driven development for your AI agent. Instead of “build authentication,” you get concrete tasks like “create a user registration endpoint that validates email format.”

4. Implement: Your coding agent tackles tasks one by one (or in parallel). Instead of reviewing thousand-line code dumps, you review focused changes that solve specific problems. The agent knows what to build (specification), how to build it (plan), and what to work on (task). Crucially, your role is to verify at each phase: Does the spec capture what you want? Does the plan account for constraints? Are there edge cases the AI missed? The process builds in checkpoints for you to critique, spot gaps, and course-correct before moving forward.

This gated workflow prevents what Willison calls “house of cards code” - fragile AI outputs that collapse under scrutiny. Anthropic’s Skills system offers a similar pattern, letting you define reusable Markdown-based behaviors that agents invoke. By embedding your spec in these workflows, you ensure the agent can’t proceed until the spec is validated, and changes propagate automatically to task breakdowns and tests.

Consider agents.md for specialized personas: For tools like GitHub Copilot, you can create agents.md files that define specialized agent personas - a @docs-agent for technical writing, a @test-agent for QA, a @security-agent for code review. Each file acts as a focused spec for that persona’s behavior, commands, and boundaries. This is particularly useful when you want different agents for different tasks rather than one general-purpose assistant.

Design for Agent Experience (AX): Just as we design APIs for developer experience (DX), consider designing specs for “Agent Experience.” This means clean, parseable formats: OpenAPI schemas for any APIs the agent will consume, llms.txt files that summarize documentation for LLM consumption, and explicit type definitions. The Agentic AI Foundation (AAIF) is standardizing protocols like MCP (Model Context Protocol) for tool integration - specs that follow these patterns are easier for agents to consume and act on reliably.

PRD vs SRS mindset: It helps to borrow from established documentation practices. For AI agent specs, you’ll often blend these into one document (as illustrated above), but covering both angles serves you well. Writing it like a PRD ensures you include user-centric context (“the why behind each feature”) so the AI doesn’t optimize for the wrong thing. Expanding it like an SRS ensures you nail down the specifics the AI will need to actually generate correct code (like what database or API to use). Developers have found that this extra upfront effort pays off by drastically reducing miscommunications with the agent later.

Make the spec a “living document”: Don’t write it and forget it. Update the spec as you and the agent make decisions or discover new info. If the AI had to change the data model or you decided to cut a feature, reflect that in the spec so it remains the ground truth. Think of it as version-controlled documentation. In spec-driven workflows, the spec drives implementation, tests, and task breakdowns, and you don’t move to coding until the spec is validated. This habit keeps the project coherent, especially if you or the agent step away and come back later. Remember, the spec isn’t just for the AI - it helps you as the developer maintain oversight and ensure the AI’s work meets the real requirements.

3. Break tasks into modular prompts and context, not one big prompt

Divide and conquer: give the AI one focused task at a time rather than a monolithic prompt with everything at once.

Experienced AI engineers have learned that trying to stuff the entire project (all requirements, all code, all instructions) into a single prompt or agent message is a recipe for confusion. Not only do you risk hitting token limits, you also risk the model losing focus due to the “curse of instructions” - too many directives causing it to follow none of them well. The solution is to design your spec and workflow in a modular way, tackling one piece at a time and pulling in only the context needed for that piece.

The curse of too much context/instructions: Research has confirmed what many devs anecdotally saw: as you pile on more instructions or data into the prompt, the model’s performance in adhering to each one drops significantly. One study dubbed this the “curse of instructions”, showing that even GPT-4 and Claude struggle when asked to satisfy many requirements simultaneously. In practical terms, if you present 10 bullet points of detailed rules, the AI might obey the first few and start overlooking others. The better strategy is iterative focus. Guidelines from industry suggest decomposing complex requirements into sequential, simple instructions as a best practice. Focus the AI on one sub-problem at a time, get that done, then move on. This keeps the quality high and errors manageable.

Divide the spec into phases or components: If your spec document is very long or covers a lot of ground, consider splitting it into parts (either physically separate files or clearly separate sections). For example, you might have a section for “Backend API Spec” and another for “Frontend UI Spec.” You don’t need to always feed the frontend spec to the AI when it’s working on the backend, and vice versa. Many devs using multi-agent setups even create separate agents or sub-processes for each part - e.g. one agent works on database/schema, another on API logic, another on frontend - each with the relevant slice of the spec. Even if you use a single agent, you can emulate this by copying only the relevant spec section into the prompt for that task. Avoid context overload: Don’t mix authentication tasks with database schema changes in one go, as the DigitalOcean AI guide warns. Keep each prompt tightly scoped to the current goal.

Extended TOC / Summaries for large specs: One clever technique is to have the agent build an extended Table of Contents with summaries for the spec. This is essentially a “spec summary” that condenses each section into a few key points or keywords, and references where details can be found. For example, if your full spec has a section on “Security Requirements” spanning 500 words, you might have the agent summarize it to: “Security: use HTTPS, protect API keys, implement input validation (see full spec §4.2)”. By creating a hierarchical summary in the planning phase, you get a bird’s-eye view that can stay in the prompt, while the fine details remain offloaded unless needed. This extended TOC acts as an index: the agent can consult it and say “aha, there’s a security section I should look at”, and you can then provide that section on demand. It’s similar to how a human developer skims an outline and then flips to the relevant page of a spec document when working on a specific part.

To implement this, you can prompt the agent after writing the spec: “Summarize the spec above into a very concise outline with each section’s key points and a reference tag.” The result might be a list of sections with one or two sentence summaries. That summary can be kept in the system or assistant message to guide the agent’s focus without eating up too many tokens. This hierarchical summarization approach is known to help LLMs maintain long-term context by focusing on the high-level structure. The agent carries a “mental map” of the spec.

Utilize sub-agents or “skills” for different spec parts: Another advanced approach is using multiple specialized agents (what Anthropic calls subagents or what you might call “skills”). Each subagent is configured for a specific area of expertise and given the portion of the spec relevant to that area. For instance, you might have a Database Designer subagent that only knows about the data model section of the spec, and an API Coder subagent that knows the API endpoints spec. The main agent (or an orchestrator) can route tasks to the appropriate subagent automatically. The benefit is each agent has a smaller context window to deal with and a more focused role, which can boost accuracy and allow parallel work on independent tasks. Anthropic’s Claude Code supports this by letting you define subagents with their own system prompts and tools. “Each subagent has a specific purpose and expertise area, uses its own context window separate from the main conversation, and has a custom system prompt guiding its behavior,” as their docs describe. When a task comes up that matches a subagent’s domain, Claude can delegate that task to it, with the subagent returning results independently.

Parallel agents for throughput: Running multiple agents simultaneously is emerging as “the next big thing” for developer productivity. Rather than waiting for one agent to finish before starting another task, you can spin up parallel agents for non-overlapping work. Willison describes this as “embracing parallel coding agents” and notes it’s “surprisingly effective, if mentally exhausting”. The key is scoping tasks so agents don’t step on each other - one agent codes a feature while another writes tests, or separate components get built concurrently. Orchestration frameworks like LangGraph or OpenAI Swarm can help coordinate these agents, and shared memory via vector databases (like Chroma) lets them access common context without redundant prompting.

Single vs. multi-agent: when to use each

AspectSingle AgentParallel/Multi-AgentStrengthsSimpler setup; lower overhead; easier to debug and followHigher throughput; handles complex interdependencies; specialists per domainChallengesContext overload on big projects; slower iteration; single point of failureCoordination overhead; potential conflicts; needs shared memory (e.g., vector DBs)Best ForIsolated modules; small-to-medium projects; early prototypingLarge codebases; one codes + one tests + one reviews; independent featuresTipsUse spec summaries; refresh context per task; start fresh sessions oftenLimit to 2-3 agents initially; use MCP for tool sharing; define clear boundaries

In practice, using subagents or skill-specific prompts might look like: you maintain multiple spec files (or prompt templates) - e.g. SPEC_backend.md, SPEC_frontend.md - and you tell the AI, “For backend tasks, refer to SPEC_backend; for frontend tasks refer to SPEC_frontend.” Or in a tool like Cursor/Claude, you actually spin up a subagent for each. This is certainly more complex to set up than a single-agent loop, but it mimics what human developers do - we mentally compartmentalize a large spec into relevant chunks (you don’t keep the whole 50-page spec in your head at once; you recall the part you need for the task at hand, and have a general sense of the overall architecture). The challenge, as noted, is managing interdependencies: the subagents must still coordinate (the frontend needs to know the API contract from the backend spec, etc.). A central overview (or an “architect” agent) can help by referencing the sub-specs and ensuring consistency.

Focus each prompt on one task/section: Even without fancy multi-agent setups, you can manually enforce modularity. For example, after the spec is written, your next move might be: “Step 1: Implement the database schema.” You feed the agent the Database section of the spec only, plus any global constraints from the spec (like tech stack). The agent works on that. Then for Step 2, “Now implement the authentication feature”, you provide the Auth section of the spec and maybe the relevant parts of the schema if needed. By refreshing the context for each major task, you ensure the model isn’t carrying a lot of stale or irrelevant information that could distract it. As one guide suggests: “Start fresh: begin new sessions to clear context when switching between major features”. You can always remind the agent of critical global rules (from the spec’s Constraints section) each time, but don’t shove the entire spec in if it’s not all needed.

Use in-line directives and code TODOs: Another modularity trick is to use your code or spec as an active part of the conversation. For instance, scaffold your code with // TODO comments that describe what needs to be done, and have the agent fill them one by one. Each TODO essentially acts as a mini-spec for a small task. This keeps the AI laser-focused (“implement this specific function according to this spec snippet”) and you can iterate in a tight loop. It’s similar to giving the AI a checklist item to complete rather than the whole checklist at once.

The bottom line: small, focused context beats one giant prompt. This improves quality and keeps the AI from getting “overwhelmed” by too much at once. As one set of best practices sums up, provide “One Task Focus” and “Relevant info only” to the model, and avoid dumping everything everywhere. By structuring the work into modules - and using strategies like spec summaries or sub-spec agents - you’ll navigate around context size limits and the AI’s short-term memory cap. Remember, a well-fed AI is like a well-fed function: give it only the inputs it needs for the job at hand.

4. Build in self-checks, constraints, and human expertise

Make your spec not just a to-do list for the agent, but also a guide for quality control - and don’t be afraid to inject your own expertise.

A good spec for an AI agent anticipates where the AI might go wrong and sets up guardrails. It also takes advantage of what you know (domain knowledge, edge cases, “gotchas”) so the AI doesn’t operate in a vacuum. Think of the spec as both coach and referee for the AI: it should encourage the right approach and call out fouls.

Use three-tier boundaries: The GitHub analysis of 2,500+ agent files found that the most effective specs use a three-tier boundary system rather than a simple list of don’ts. This gives the agent clearer guidance on when to proceed, when to pause, and when to stop:

✅ Always do: Actions the agent should take without asking. “Always run tests before commits.” “Always follow the naming conventions in the style guide.” “Always log errors to the monitoring service.”

⚠️ Ask first: Actions that require human approval. “Ask before modifying database schemas.” “Ask before adding new dependencies.” “Ask before changing CI/CD configuration.” This tier catches high-impact changes that might be fine but warrant a human check.

🚫 Never do: Hard stops. “Never commit secrets or API keys.” “Never edit node_modules/ or vendor/.” “Never remove a failing test without explicit approval.” “Never commit secrets” was the single most common helpful constraint in the study.

This three-tier approach is more nuanced than a flat list of rules. It acknowledges that some actions are always safe, some need oversight, and some are categorically off-limits. The agent can proceed confidently on “Always” items, flag “Ask first” items for review, and hard-stop on “Never” items.

Encourage self-verification: One powerful pattern is to have the agent verify its work against the spec automatically. If your tooling allows, you can integrate checks like unit tests or linting that the AI can run after generating code. But even at the spec/prompt level, you can instruct the AI to double-check: e.g. “After implementing, compare the result with the spec and confirm all requirements are met. List any spec items that are not addressed.” This pushes the LLM to reflect on its output relative to the spec, catching omissions. It’s a form of self-audit built into the process.

For instance, you might append to a prompt: “(After writing the function, review the above requirements list and ensure each is satisfied, marking any missing ones).” The model will then (ideally) output the code followed by a short checklist indicating if it met each requirement. This reduces the chance it forgets something before you even run tests. It’s not foolproof, but it helps.

LLM-as-a-Judge for subjective checks: For criteria that are hard to test automatically - code style, readability, adherence to architectural patterns - consider using “LLM-as-a-Judge.” This means having a second agent (or a separate prompt) review the first agent’s output against your spec’s quality guidelines. Anthropic and others have found this effective for subjective evaluation. You might prompt: “Review this code for adherence to our style guide. Flag any violations.” The judge agent returns feedback that either gets incorporated or triggers a revision. This adds a layer of semantic evaluation beyond syntax checks.

Conformance testing: Willison advocates building conformance suites - language-independent tests (often YAML-based) that any implementation must pass. These act as a contract: if you’re building an API, the conformance suite specifies expected inputs/outputs, and the agent’s code must satisfy all cases. This is more rigorous than ad-hoc unit tests because it’s derived directly from the spec and can be reused across implementations. Include conformance criteria in your spec’s Success section (e.g., “Must pass all cases in conformance/api-tests.yaml”).

Leverage testing in the spec: If possible, incorporate a test plan or even actual tests in your spec and prompt flow. In traditional development, we use TDD or write test cases to clarify requirements - you can do the same with AI. For example, in the spec’s Success Criteria, you might say “These sample inputs should produce these outputs…” or “the following unit tests should pass.” The agent can be prompted to run through those cases in its head or actually execute them if it has that capability. Simon Willison noted that having a robust test suite is like giving the agents superpowers - they can validate and iterate quickly when tests fail. In an AI coding context, writing a bit of pseudocode for tests or expected outcomes in the spec can guide the agent’s implementation. Additionally, you can use a dedicated “test agent” in a subagent setup that takes the spec’s criteria and continuously verifies the “code agent’s” output.

Bring your domain knowledge: Your spec should reflect insights that only an experienced developer or someone with context would know. For example, if you’re building an e-commerce agent and you know that “products” and “categories” have a many-to-many relationship, state that clearly (don’t assume the AI will infer it - it might not). If a certain library is notoriously tricky, mention pitfalls to avoid. Essentially, pour your mentorship into the spec. The spec can contain advice like “If using library X, watch out for memory leak issue in version Y (apply workaround Z).” This level of detail is what turns an average AI output into a truly robust solution, because you’ve steered the AI away from common traps.

Also, if you have preferences or style guidelines (say, “use functional components over class components in React”), encode that in the spec. The AI will then emulate your style. Many engineers even include small examples in the spec, e.g., “All API responses should be JSON. E.g. {“error”: “message”} for errors.” By giving a quick example, you anchor the AI to the exact format you want.

Minimalism for simple tasks: While we advocate thorough specs, part of expertise is knowing when to keep it simple. For relatively simple, isolated tasks, an overbearing spec can actually confuse more than help. If you’re asking the agent to do something straightforward (like “center a div on the page”), you might just say, “Make sure to keep the solution concise and do not add extraneous markup or styles.” No need for a full PRD there. Conversely, for complex tasks (like “implement an OAuth flow with token refresh and error handling”), that’s when you break out the detailed spec. A good rule of thumb: adjust spec detail to task complexity. Don’t under-spec a hard problem (the agent will flail or go off-track), but don’t over-spec a trivial one (the agent might get tangled or use up context on unnecessary instructions).

Maintain the AI’s “persona” if needed: Sometimes, part of your spec is defining how the agent should behave or respond, especially if the agent interacts with users. For example, if building a customer support agent, your spec might include guidelines like “Use a friendly and professional tone,” “If you don’t know the answer, ask for clarification or offer to follow up, rather than guessing.” These kind of rules (often included in system prompts) help keep the AI’s outputs aligned with expectations. They are essentially spec items for AI behavior. Keep them consistent and remind the model of them if needed in long sessions (LLMs can “drift” in style over time if not kept on a leash).

You remain the exec in the loop: The spec empowers the agent, but you remain the ultimate quality filter. If the agent produces something that technically meets the spec but doesn’t feel right, trust your judgement. Either refine the spec or directly adjust the output. The great thing about AI agents is they don’t get offended - if they deliver a design that’s off, you can say, “Actually, that’s not what I intended, let’s clarify the spec and redo it.” The spec is a living artifact in collaboration with the AI, not a one-time contract you can’t change.

Simon Willison humorously likened working with AI agents to “a very weird form of management” and even “getting good results out of a coding agent feels uncomfortably close to managing a human intern”. You need to provide clear instructions (the spec), ensure they have the necessary context (the spec and relevant data), and give actionable feedback. The spec sets the stage, but monitoring and feedback during execution are key. If an AI was a “weird digital intern who will absolutely cheat if you give them a chance”, the spec and constraints you write are how you prevent that cheating and keep them on task.

Here’s the payoff: a good spec doesn’t just tell the AI what to build, it also helps it self-correct and stay within safe boundaries. By baking in verification steps, constraints, and your hard-earned knowledge, you drastically increase the odds that the agent’s output is correct on the first try (or at least much closer to correct). This reduces iterations and those “why on Earth did it do that?” moments.

5. Test, iterate, and evolve the spec (and use the right tools)

Think of spec-writing and agent-building as an iterative loop: test early, gather feedback, refine the spec, and leverage tools to automate checks.

The initial spec is not the end - it’s the beginning of a cycle. The best outcomes come when you continually verify the agent’s work against the spec and adjust accordingly. Also, modern AI devs use various tools to support this process (from CI pipelines to context management utilities).

Continuous testing: Don’t wait until the end to see if the agent met the spec. After each major milestone or even each function, run tests or at least do quick manual checks. If something fails, update the spec or prompt before proceeding. For example, if the spec said “passwords must be hashed with bcrypt” and you see the agent’s code storing plain text - stop and correct it (and remind the spec or prompt about the rule). Automated tests shine here: if you provided tests (or write them as you go), let the agent run them. In many coding agent setups, you can have an agent run npm test or similar after finishing a task. The results (failures) can then feed back into the next prompt, effectively telling the agent “your output didn’t meet spec on X, Y, Z - fix it.” This kind of agentic loop (code -> test -> fix -> repeat) is extremely powerful and is how tools like Claude Code or Copilot Labs are evolving to handle larger tasks. Always define what “done” means (via tests or criteria) and check for it.

Iterate on the spec itself: If you discover that the spec was incomplete or unclear (maybe the agent misunderstood something or you realized you missed a requirement), update the spec document. Then explicitly re-sync the agent with the new spec: “I have updated the spec as follows… Given the updated spec, adjust the plan or refactor the code accordingly.” This way the spec remains the single source of truth. It’s similar to how we handle changing requirements in normal dev - but in this case you’re also the product manager for your AI agent. Keep version history if possible (even just via commit messages or notes), so you know what changed and why.

Utilize context-management and memory tools: There’s a growing ecosystem of tools to help manage AI agent context and knowledge. For instance, retrieval-augmented generation (RAG) is a pattern where the agent can pull in relevant chunks of data from a knowledge base (like a vector database) on the fly. If your spec is huge, you could embed sections of it and let the agent retrieve the most relevant parts when needed, instead of always providing the whole thing. There are also frameworks implementing the Model Context Protocol (MCP), which automates feeding the right context to the model based on the current task. One example is Context7 (context7.com), which can auto-fetch relevant context snippets from docs based on what you’re working on. In practice, this might mean the agent notices you’re working on “payment processing” and it pulls the “Payments” section of your spec or documentation into the prompt. Consider leveraging such tools or setting up a rudimentary version (even a simple search in your spec document).

Parallelize carefully: Some developers run multiple agent instances in parallel on different tasks (as mentioned earlier with subagents). This can speed up development - e.g., one agent generates code while another simultaneously writes tests, or two features are built concurrently. If you go this route, ensure the tasks are truly independent or clearly separated to avoid conflicts (the spec should note any dependencies). For example, don’t have two agents writing to the same file at once. One workflow is to have an agent generate code and another review it in parallel, or to have separate components built that integrate later. This is advanced usage and can be mentally taxing to manage (as Willison admitted, running multiple agents is surprisingly effective, if mentally exhausting!). Start with at most 2-3 agents to keep things manageable.

Version control and spec locks: Use Git or your version control of choice to track what the agent does. Good version control habits matter even more with AI assistance. Commit the spec file itself to the repo. This not only preserves history, but the agent can even use git diff or blame to understand changes (LLMs are quite capable of reading diffs). Some advanced agent setups let the agent query the VCS history to see when something was introduced - surprisingly, models can be “fiercely competent at Git”. By keeping your spec in the repo, you allow both you and the AI to track evolution. There are tools (like GitHub Spec Kit mentioned earlier) that integrate spec-driven development into the git workflow - for instance, gating merges on updated specs or generating checklists from spec items. While you don’t need those tools to succeed, the takeaway is to treat the spec like code - maintain it diligently.

Cost and speed considerations: Working with large models and long contexts can be slow and expensive. A practical tip is to use model selection and batching smartly. Perhaps use a cheaper/faster model for initial drafts or repetitions, and reserve the most capable (and expensive) model for final outputs or complex reasoning. Some developers use GPT-4 or Claude for planning and critical steps, but offload simpler expansions or refactors to a local model or a smaller API model. If using multiple agents, maybe not all need to be top-tier; a test-running agent or a linter agent could be a smaller model. Also consider throttling context size: don’t feed 20k tokens if 5k will do. As we discussed, more tokens can mean diminishing returns.

Monitor and log everything: In complex agent workflows, logging the agent’s actions and outputs is essential. Check the logs to see if the agent is deviating or encountering errors. Many frameworks provide trace logs or allow printing the agent’s chain-of-thought (especially if you prompt it to think step-by-step). Reviewing these logs can highlight where the spec or instructions might have been misinterpreted. It’s not unlike debugging a program - except the “program” is the conversation/prompt chain. If something weird happens, go back to the spec/instructions to see if there was ambiguity.

Learn and improve: Finally, treat each project as a learning opportunity to refine your spec-writing skill. Maybe you’ll discover that a certain phrasing consistently confuses the AI, or that organizing spec sections in a certain way yields better adherence. Incorporate those lessons into the next spec. The field of AI agents is rapidly evolving, so new best practices (and tools) emerge constantly. Stay updated via blogs (like the ones by Simon Willison, Andrej Karpathy, etc.), and don’t hesitate to experiment.

A spec for an AI agent isn’t “write once, done.” It’s part of a continuous cycle of instructing, verifying, and refining. The payoff for this diligence is substantial: by catching issues early and keeping the agent aligned, you avoid costly rewrites or failures later. As one AI engineer quipped, using these practices can feel like having “an army of interns” working for you, but you have to manage them well. A good spec, continuously maintained, is your management tool.

Avoid common pitfalls

Before wrapping up, it’s worth calling out anti-patterns that can derail even well-intentioned spec-driven workflows. The GitHub study of 2,500+ agent files revealed a stark divide: “Most agent files fail because they’re too vague.” Here are the mistakes to avoid:

Vague prompts: “Build me something cool” or “Make it work better” gives the agent nothing to anchor on. As Baptiste Studer puts it: “Vague prompts mean wrong results.” Be specific about inputs, outputs, and constraints. “You are a helpful coding assistant” doesn’t work. “You are a test engineer who writes tests for React components, follows these examples, and never modifies source code” does.

Overlong contexts without summarization: Dumping 50 pages of documentation into a prompt and hoping the model figures it out rarely works. Use hierarchical summaries (as discussed in Principle 3) or RAG to surface only what’s relevant. Context length is not a substitute for context quality.

Skipping human review: Willison has a personal rule: “I won’t commit code I couldn’t explain to someone else.” Just because the agent produced something that passes tests doesn’t mean it’s correct, secure, or maintainable. Always review critical code paths. The “house of cards” metaphor applies: AI-generated code can look solid but collapse under edge cases you didn’t test.

Conflating vibe coding with production engineering: Rapid prototyping with AI (“vibe coding”) is great for exploration and throwaway projects. But shipping that code to production without rigorous specs, tests, and review is asking for trouble. I distinguish “vibe coding” from “AI-assisted engineering” - the latter requires the discipline this guide describes. Know which mode you’re in.

Ignoring the “lethal trifecta”: Willison warns of three properties that make AI agents dangerous: speed (they work faster than you can review), non-determinism (same input, different outputs), and cost (encouraging corner-cutting on verification). Your spec and review process must account for all three. Don’t let speed outpace your ability to verify.

Missing the six core areas: If your spec doesn’t cover commands, testing, project structure, code style, git workflow, and boundaries, you’re likely missing something the agent needs. Use the six-area checklist from Section 2 as a sanity check before handing off to the agent.

Conclusion

Writing an effective spec for AI coding agents requires solid software engineering principles combined with adaptation to LLM quirks. Start with clarity of purpose and let the AI help expand the plan. Structure the spec like a serious design document - covering the six core areas and integrating it into your toolchain so it becomes an executable artifact, not just prose. Keep the agent’s focus tight by feeding it one piece of the puzzle at a time (and consider clever tactics like summary TOCs, subagents, or parallel orchestration to handle big specs). Anticipate pitfalls by including three-tier boundaries (Always/Ask first/Never), self-checks, and conformance tests - essentially, teach the AI how to not fail. And treat the whole process as iterative: use tests and feedback to refine both the spec and the code continuously.

Follow these guidelines and your AI agent will be far less likely to “break down” under large contexts or wander off into nonsense.

Happy spec-writing!

I’m excited to share I’ve released a new AI-assisted engineering book with O’Reilly. There are a number of free tips on the book site in case interested.

If your pull request doesn’t contain evidence that it works, you’re not shipping faster - you’re just moving work downstream.

By early 2026, over 30% of senior developers report shipping mostly AI-generated code. The challenge? AI excels at drafting features but falters on logic, security, and edge cases - making errors 75% more common in logic alone. This splits workflows: solos “vibe” at inference speed with test suites as backstops, while teams demand human eyes for context and compliance. Done right, both treat AI as an accelerator, but verification - who, what, and when - defines the difference.

As I’ve said before: if you haven’t seen the code do the right thing yourself, it doesn’t work. AI amplifies this rule, not excuses it.

How developers use AI for review

• Ad-hoc LLM checks: Paste diffs into Claude, Gemini or GPT for quick bug/style scans before committing.

• IDE integrations: Tools like Cursor, Claude Code, or Gemini CLI for inline suggestions and refactors during coding.

• PR bots and scanners: GitHub Copilot or custom agents to flag issues in PRs; pair with static/dynamic analysis like Snyk for security.

• Automated testing loops: Use AI to generate and run tests, enforcing coverage >70% as a gate.

• Multi-model reviews: Run code through different LLMs (e.g., Claude for generation, a security-focused model for audit) to catch biases.

The workflow and mindset differ dramatically depending on whether you’re solo or working in a team where others maintain your code.

Solo vs. Team: A quick comparison

Solo Devs: Shipping at “inference speed”

Solo developers increasingly “trust the vibe” of AI-generated code - shipping features rapidly by reviewing only the key parts and relying on tests to catch issues.

This workflow treats coding agents as powerful interns that can handle massive refactors largely on their own. As Peter Steinberger admits: “I don’t read much code anymore. I watch the stream and sometimes look at key parts, but most code I don’t read.” The bottleneck becomes inference time - waiting for the AI to generate output - not typing.

There’s a catch: perceived speed gains vanish without strong testing practices. Build those first. If you skip review, you don’t eliminate work - you defer it. The developers who succeed with AI at high velocity aren’t the ones who blindly trust it; they’re the ones who’ve built verification systems that catch issues before they reach production.

That isn’t to say solos throw caution to the wind. The responsible ones employ extensive automated testing as a safety net - aiming for high coverage (often >70%) and using AI to generate tests that catch bugs in real-time. Modern coding agents are surprisingly good at designing sophisticated end-to-end tests.

For solos, the game-changer is language-independent, data-driven tests. If comprehensive, they let an agent build (or fix) implementations in any language, verifying as it goes. I start projects with a spec.md the AI drafts, approve it, then loop: write → test → fix.

Crucially, solo coders still do manual testing and critical reasoning on the final product. Run the application, click through the UI, use the feature yourself. When higher stakes are involved, read more code and add extra checks. And despite moving fast, fix ugly code when you see it rather than letting the mess accumulate.

Even in this bleeding-edge paradigm: your job is to deliver code you have proven to work.

Teams: AI shifts review bottlenecks

In team settings, AI is a powerful assistant for code review, but cannot replace the human judgment needed for quality, security, and maintainability.

When multiple engineers collaborate, the cost of mistakes and longevity of code are much higher concerns. Teams have started using AI-based review bots for an initial pass on PRs, but they still require a human to sign off. As Greg Foster of Graphite puts it: “I don’t ever see [AI agents] becoming a stand-in for an actual human engineer signing off on a pull request.”

The biggest practical problem isn’t that AI reviewers miss style issues - it’s that AI increases volume and shifts the burden onto humans. PRs are getting larger (~18% more additions as AI adoption increases), incidents per PR are up ~24%, and change failure rates up ~30%. When output increases faster than verification capacity, review becomes the rate limiter. As Foster notes: “If we’re shipping code that’s never actually read or understood by a fellow human, we’re running a huge risk.”

In teams, AI floods volume, so enforce incrementalism: break agent output into digestible commits. Human sign-off isn’t going away - it’s evolving to focus on what AI misses, like roadmap alignment and institutional context that AI can’t grasp.

Security: AI’s predictable weaknesses

One area where human oversight is absolutely non-negotiable is security. Approximately 45% of AI-generated code contains security flaws. Logic errors appear at 1.75× the rate of human-written code, and XSS vulnerabilities occur at 2.74× higher frequency.

Beyond code issues, agentic tooling and AI-integrated IDEs have created new attack paths - prompt injection, data exfiltration, even RCE vulnerabilities. AI expands attack surfaces, so hybrid approaches win: AI flags, humans verify.

Rule: If code touches auth, payments, secrets, or untrusted input, treat AI as a high-speed intern and require a human threat model review plus a security tool pass before merge.

Review as knowledge transfer

Code review is also how teams share system context. If AI writes the code and nobody can explain it, on-call becomes expensive.

When a developer submits AI-generated code they don’t fully understand, they’re breaking the knowledge transfer mechanism that makes teams resilient. If the original author can’t explain why the code works, how will the on-call engineer debug it at 2 AM?

The OCaml maintainers’ rejection of a 13,000-line AI-generated PR crystallizes this issue. The code wasn’t necessarily bad, but no one had bandwidth to review such a huge change, and reviewing AI-generated code is “more taxing” than reviewing human code. The lesson: AI can flood you with code, but teams must manage volume to avoid a review bottleneck.

Making AI review tools work

User experiences with AI review tools are decidedly mixed. On the positive side, teams report catching 95%+ of bugs in some cases - null pointer exceptions, missing test coverage, anti-patterns. On the negative side, some developers dismiss AI review comments as “text noise” - generic observations that add no value.

The lesson: AI review tools require thoughtful configuration. Tune sensitivity levels, disable unhelpful comment types, and establish clear opt-in/opt-out policies. Properly configured, AI reviewers can catch 70-80% of low-hanging fruit, freeing humans to focus on architecture and business logic.

Many teams encourage smaller, stackable pull requests even if AI could do a giant change all at once. Commit early and often - treat each self-contained change as a separate commit/PR with clear messages.

Importantly, teams maintain a hard line of human accountability. No matter how much AI contributed, a human must take responsibility. As an old IBM training saying goes: “A computer can never be held accountable. That’s your job as the human in the loop.”

The PR Contract: What authors owe reviewers

Whether solo or in a team, the emerging best practice is to treat AI-generated code as a helpful draft that must be verified.

The most successful teams have converged on a simple framework:

PR Contract

1. What/why: Intent in 1-2 sentences.

2. Proof it works: Tests passed, manual steps (screenshots/logs).

3. Risk + AI role: Tier and which parts were AI-generated (e.g., high=payments).

4. Review focus: 1-2 areas for human input (e.g., architecture).

This isn’t bureaucracy - it’s respect for reviewer time and a forcing function for author accountability. If you can’t fill this out, you don’t understand your own change well enough to ask someone else to approve it.

Core Principles

Insist on proof, not promises. Make “working code” the baseline. Prompt AI agents to execute code or run unit tests after generation. Demand evidence: logs, screenshots, results. No PR goes up without either new tests or a demo of the change working.

Use AI as first-pass reviewer, not final arbiter. Treat AI review output as advisory - a dialog where one AI writes code, another reviews it, and the human orchestrates fixes. Think of AI reviews as spellcheck, not an editor.

Focus human review on what AI misses. Does the change introduce a security hole? Does it duplicate existing code (a common AI flaw)? Is the approach maintainable? AI triages the easy stuff; humans tackle the hard stuff.

Enforce incremental development. Break work into small pieces - easier for AI to produce and for humans to review. Small commits with clear messages serve as checkpoints. Never commit code you can’t explain.

Maintain high testing standards. Those who get the most out of coding agents have strong testing practices. Ask AI to draft tests - it’s good at generating edge-case tests you might not think of.

Looking Ahead: The bottleneck has moved

AI is transforming code review from line-by-line gatekeeping into higher-level quality control - but human judgment remains the safety-critical component.

What we’re seeing is workflow evolution, not elimination. Code reviews now involve reviewing a conversation or plan between AI and author as much as the code diff itself. The human reviewer’s role becomes more like an editor or architect: focusing on what’s important and trusting automation for mundane checks.

For solo developers, the path ahead is exhilarating - new tools will further streamline development. Even then, the wise developer will “trust but verify.”

In larger teams, expect growing emphasis on AI governance. Companies will formalize policies about AI contributions, requiring sign-offs that code was reviewed by an employee. Roles like “AI code auditor” will emerge. Enterprise platforms will evolve to offer better multi-repository context and custom policy enforcement.

No matter the advances, the core principle remains: code review ensures software meets requirements, is secure, robust, and maintainable. AI doesn’t change those fundamentals - it just changes how we get there.

The bottleneck moved from writing code to proving it works. The best code reviewers in the age of AI will embrace this shift - letting AI accelerate the mechanical work while holding the line on accountability. They’ll let AI accelerate the process, never abdicate it. As engineers are learning, it’s about “proof over vibes” in coding.

Code review isn’t dead but it’s becoming more strategic. And whether you’re a solo hacker deploying at 2 AM or a team lead signing off a critical system change, one truth holds: the human is ultimately responsible for what the AI delivers.

Embrace the AI, but never forget to double-check the work.

I’m excited to share I’ve released a new AI-assisted engineering book with O’Reilly. There are a number of free tips on the book site in case interested.

This article is based on my closing keynote at React Summit by GitNation (video).

The Question everyone’s asking (but nobody’s answering well)

Let me be direct: most conversations about AI and coding are stuck on vibes. Either AI is magic that will replace us all, or it’s garbage that can’t do anything useful, or it’s perpetually “one prompt away” from shipping production code. All three takes miss what’s actually interesting.

After spending a year analyzing benchmarks, building with these tools at Google, and watching React developers struggle (and succeed) with AI assistants, here’s what I’ve learned: AI is already useful for React developers, but its usefulness is extremely uneven. The unevenness is predictable if you know what to look for.

More importantly - and this is the part most articles skip - you have far more control over outcomes than you think.

The Core Thesis: Two sides of the same coin

This article covers two critical angles:

What the data tells us: Benchmarks like Design Arena, Web Dev Arena, SWE-Bench, and Web-Bench reveal clear patterns about where AI excels (isolated components, scaffolding, implementing explicit requirements) and where it struggles (multi-step integration, design taste, complex state management). Understanding these patterns means you can predict what will work before you waste time.

What you can control: The difference between “AI helped me ship” and “AI gave me a mess to untangle” almost never comes down to just model selection. It comes down to context engineering, prompt specificity, workflow structure, and guardrails. These are all in your power to fix.

Let’s start with the foundation.

90% of developers use AI for coding in some way. In an AI-assisted world, the value of frameworks like React depends on how effectively AI can use them. If AI can’t handle a framework well, that means the quality of experiences you can build without a lot of manual work can be limited.

A lot of AI code quality comes down to context. You want to squeeze the most value out of the token budget in your context window. But the model, the tools that sit on top of it all of these layers play an important role.

AI Changes what is easy, not what is true

AI is a force multiplier. It amplifies everything: good requirements, good architecture, good taste. It also amplifies the bad: vague specs, messy state, and the temptation to ship something you haven’t really understood. Give it a weak brief, and it will happily hand you a 10,000-line maze you’ll later delete.

I draw a hard line between AI-assisted vibe coding and AI-assisted engineering. Vibe coding is trusting high-level prompts and prioritizing speed over review. AI-assisted engineering is integrating AI inside a structured process where the human stays in control and accountable for the output.

Why does this distinction matter for React?

Because React apps aren’t just code. They’re product behavior, user experience, reliability, security, performance, accessibility, and long-term maintenance. AI can help with all of that - but only if you treat it like a teammate you’re pairing with and have oversight over, not a vending machine dispensing code.

The Monoculture problem (and opportunity)

One of the most under-discussed parts of the AI coding story: “how well AI codes” is not a universal property. It depends on what the model has seen in training, what tools it has access to, and what the ecosystem has standardized on.

Large language models effectively set the ceiling for how much leverage you get out of a framework in an AI-assisted workflow. If AI struggles with a framework, you feel that as friction and quality limits.

Most AI tools converge on a stack that looks like: React, TypeScript, Tailwind, shadcn/ui. That stack dominates training data and tool optimization, so models are competent there and noticeably shakier off the beaten path.

This has two implications for practicing React developers:

1. If you’re on the mainstream stack, your “AI assistance ceiling” is higher. You’ll get better scaffolds, better component generation, and fewer hallucinated APIs.

2. If you’re not, you need to compensate with better context, doc retrieval, and stricter constraints - or you’ll watch the model confidently build an alternate universe version of your app.

There’s also a second-order effect: monoculture can slow innovation. React skills are likely to stay very relevant (comforting), but it also means newer frameworks or alternative patterns can face headwinds until models and tools catch up.

The good news? If a framework gains traction, AI makers will fine-tune models on it. Docs MCPs can bridge gaps in the interim. But short-term, React’s position is extremely strong because AI “knows” it best.

The Big reality check: The complexity cliff

If you remember nothing else from this article, remember this: AI handles simple tasks well and then falls off a cliff as complexity rises.

A form component, a utility function, a small isolated widget: great. Multi-step work across a real codebase: much less reliable.

I used a mix of objective and human-rated benchmarks to show this pattern. We need both, because pass/fail benchmarks tell you “can it solve the issue,” while human-rated arenas tell you something equally important for frontend work: “do humans actually want to use what it builds.”

What The Numbers Show

On objective benchmarks, the complexity cliff is visible:

• Next.js eval tasks: Best models around 42% success - roughly 21 of 50 tasks completed. Even on framework-specific challenges, failures are common.

• Web-Bench multi-step full-stack tasks: Around 25% tasks solved. Many failures as steps chain together.

• SWE-Bench Pro: Around 20-43% on the Pro public set, versus jumping to over 70% on SWE-Bench Verified. Increasing complexity collapses performance.

The gap between benchmark performance and your real codebase is the important thing to calibrate to.

My practical translation of the complexity cliff for React developers:

• AI is great at first drafts

• AI is mediocre at integration

• AI is unreliable at long multi-step changes unless you give it strong tooling and context

• AI gets you to “it works” faster than it gets you to “it’s a codebase I want to own”

Design Arena and Web Dev Arena: Where React developers should pay attention

React developers spend a lot of time in the space between “the code runs” and “this is good.” That space includes UI quality, hierarchy, spacing, accessibility, and whether the end result feels intentional.

Design Arena is interesting because it’s explicitly human preference–driven. Here’s how it works:

• Users come to the platform to explore and use the best AI-powered tools, like website generation and game generation

• Design Arena presents multiple versions of the same experience (a website, agent, or builder), and users can save their favorite

• Rankings emerge from these aggregated choices across categories like website generation, agents, and builders, reflecting real usage preferences rather than curated rubrics

• Elo-style scores are calculated using a Bradley–Terry model, with models below a minimum comparison threshold filtered out.

• The leaderboard updates live (every three hours, per their methodology), powered by interactions from over 850,000 users across 145 countries.

By using a pairwise comparison system, DesignArena generates leaderboards that rank AI models based on human preferences, helping to measure and drive improvements in design quality, usability, and aesthetics.

Similarly, Web Dev Arena is an open-source benchmarking platform from LMArena designed to evaluate LLMs based on their capability to build functional, interactive web applications. Users submit a prompt and compare anonymous AI models generating code side-by-side, contributing to a community-driven Elo leaderboard that ranks top models for complex web development tasks.

So what do React developers learn from such arenas?

The Core Finding: AI has mastered logic, but not taste

This slide is the thesis of the whole talk. Models can solve hard reasoning problems and still produce UIs with basic design failures: off color choices, inconsistent spacing, weak hierarchy.

I call this the capability divide:

• AI is strong at logic, data flow, and implementing explicit requirements

• AI is weak at taste, usability awareness, and aesthetic judgment

If you’re a React developer, this should change how you delegate:

• Delegate boilerplate and mechanical implementation

• Keep design intent, API design, and architecture decisions human-led

• Treat “pretty” as an explicit requirement, not a default outcome

The Surprise: Tools and scaffolding matter more than you think

Design Arena also found something counterintuitive: general agents are more variable than specialists, and the scaffolding and workflow around the base model drives a lot of the performance spread.

Put differently: two products can wrap the same base model and feel wildly different because of tooling, retrieval, iteration loops, and guardrails.

This is great news, because it means you have leverage even when you don’t control the base model.

Arena by Arena: What React developers should steal from the data

Let me walk through five arenas and extract the practical lessons for each one.

1. Website Arena: Prompt to website (and why Purple keeps happening)

The Website Arena measures how well models generate complete single-page sites from a prompt, with instructions to add modern UI/UX practices, accessibility, and responsive design.

The important nuance is how winners tend to win: it’s not always the flashiest layout, it’s often the most coherent and complete page.

If your goal is something shippable, bias your prompts toward coherence and structure, not 'make it look cool.

Why is there so much purple?

I joked about this in the talk because once you see it, you can’t unsee it: models converge on safe, generic design patterns, and “purple gradient plus glassmorphism” is one of those defaults.

That’s not just a meme. It’s distributional convergence: under uncertainty, models gravitate toward common patterns in the data.

How do you fix it?

One approach is tooling rather than “better prompts forever.” Anthropic pushed some of this into Skills (markdown files that Claude reads on demand) instead of trying to brute force it through training. Their frontend-design skill is worth checking out.

Even if you never use Claude Skills specifically, the lesson is broader:

• Some failures are better solved by scaffolding and constraints than by model selection

• You want repeatable, shareable “taste primitives” that don’t require rewriting your entire prompt every time

My website generation checklist for React teams

What I ask for up front:

• Anchor the layout first: Specify the page sections you want before code

• Specify stack and routing: Call out Next App Router, file names, and RSC vs client components so it doesn’t invent structure

• Describe content density: Minimal landing page vs long-form so spacing doesn’t default to sludge

• Ask for responsive constraints: Breakpoints and collapse behavior

• Bake in accessibility: Semantic landmarks, skip links, labels, safe contrast

• Convert HTML to real React files: Map sections to components and wire them up in page.tsx

What I do after generation, before trusting it:

• Strip inline scripts and move DOM logic into client components with hooks and typed props

• Normalize layout primitives and refactor div soup into your real Shell, Container, Stack components

• Run a11y and perf checks: Lint, Lighthouse, and add tests for critical flows

• Freeze the visual system: Snap palette, spacing, typography into Tailwind config or tokens

• Keep the model on a leash: Use it for slices and variants, not wholesale rewrites of a tuned page

Single sentence summary: Be radically explicit in your instructions, and enforce your design system and coding standards so the model can’t drift.

Poor prompt:

Make a landing page for a SaaS product

Strong prompt:

Create a Next.js App Router landing page (app/page.tsx) for a developer tools SaaS:

Layout sections:
1. Hero with headline, subheadline, CTA
2. Features (3 columns, icon + title + description each)
3. Social proof (logos grid)
4. CTA

Stack: Next.js 15, TypeScript, Tailwind
Density: Spacious landing page (not cramped)
Colors: Avoid purple/pink gradients - use neutral gray with blue accent
Responsive: Stack features vertically below 768px

Accessibility:
- Semantic HTML (header, main, section)
- Alt text for all images
- Sufficient color contrast (WCAG AA)

2. Agent Arena: Most failures are context failures now

The Agent Arena is a step up: multi-step tasks like writing code, fixing bugs, running tests, running browsers, debugging. This is where “agent loops” show up.

Here’s the biggest trap: when agents fail, it often looks like “the model is dumb.” Increasingly, that’s not true.

Most agent failures are context failures. If the agent doesn’t see the right logs, tests, or constraints, it makes confident but wrong changes. Fixing context is often higher leverage than switching models.

I also called out something that will resonate if you’ve ever spent time tweaking prompts: prompt engineering failures often come from context mismanagement, not “the wrong magic words.”

How I run agents like a React team lead

Treat agents like a junior hire:

• Give a written task brief, acceptance criteria, and constraints

• Declare the sandbox: disposable branch, test DB, temporary env vars

• Ask for a plan first: files it will touch, tools it will call, risks it sees

• Cap blast radius: constrain write access to app, src, config

• Require tests as part of fixes: reproduce bug first, then patch

• Force small PRs: reviewable commits, not a mega diff

Then add operational guardrails:

• Point them at logs and monitors: build logs, Sentry traces, Playwright failures

• Snap to house style: ESLint config, prettier rules, naming conventions

• Disable auto-merge and require human approval for agent changes

That’s the difference between “agentic coding” and “outsourcing your codebase to a stochastic parrot.”

Poor prompt:

Fix the bug in the checkout flow

Strong prompt:

Task: Fix abandoned cart bug in checkout

Context:
- File: app/checkout/page.tsx
- Error: Cart resets on page refresh
- Expected: Cart persists via localStorage
- Test: Run `npm test checkout.test.tsx` to verify

Plan required before implementation:
1. Identify where cart state is managed
2. Add localStorage persistence
3. Add hydration logic
4. Update tests
5. Verify in Playwright

Constraints:
- Only modify app/checkout/* and lib/cart.ts
- Maintain existing TypeScript types
- Follow our ESLint rules

3. Context Engineering: The highest leverage skill for agentic React

If context is the bottleneck, then context engineering is the discipline.

In the talk I described context engineering as the art and science of filling the context window with just the right information to guide the agent’s performance. It’s more than clever prompting.

Two specific tips I want most React developers to internalize:

1. Visual context is powerful. Screenshots can enable one-shot solutions for UI bugs or design tasks.

2. Structure beats volume. Unstructured dumps confuse the model, competing information distracts it, and overload overwhelms it.

Under the hood, this ties back to a core principle: “Find the smallest possible set of high-signal tokens that maximize the likelihood of your desired outcome.”

Every token you waste is context you cannot spend on:

• The actual API surface you need

• The architectural constraints you care about

• The failing test output that would prevent a bad patch

4. Tooling: If you can’t control the base model, control the layer around It

As I said in the “mastering the tools” section: you probably don’t control the base model, but you can absolutely steer the tooling around it.

A concrete example is doc and state retrieval. Let me show you two tools that demonstrate this pattern:

Context7 MCP

Context7 pulls fresh, version-specific docs and examples from source sites and injects them into the model’s working set, reducing guessing and stale snippets. You can nudge it toward topics like routing or hooks and cap how much to bring in.

Next.js DevTools MCP

The modern Next dev server exposes a built-in MCP endpoint. The Next.js DevTools MCP server connects to it so an agent can ask for real data about your running app:

• Current build or runtime errors

• Routes and layouts

• Component metadata

• Server actions and dev logs

• Playwright paths for simple browser checks

It also ships with a Next-specific knowledge base and helpers for common tasks like upgrades.

Chrome DevTools MCP

Chrome DevTools MCP gives the agent eyes and hands in a real browser. It can open pages, click through flows, read console and network logs, take screenshots, and record performance traces to investigate things like high LCP or blocking time. Under the hood it rides on Chrome DevTools and Puppeteer, so you get reliable automation instead of brittle scripts. Because it can see page content, you still want sensible flags and isolation from personal browsing, but treated as scoped tooling it is very powerful.

How MCPs fit together

Context7 gives your assistant the right external knowledge. Next DevTools MCP gives it your app’s truth. Chrome DevTools MCP proves the result in a real browser. Used together, you turn a guessing assistant into a closed‑loop coder and debugger that cites sources, places changes correctly, and verifies outcomes before you hit commit.

This is the pattern I expect more React teams to adopt: rather than hoping the model remembers today’s Next.js behavior, wire it to an always-correct source of truth.

5. Builder Arena: Vibe Coding tools used responsibly

Builder tools are designed for rapid prompt-driven product creation, not just “write me a component.” They optimize for cohesion and perceived completeness.

Design Arena’s builder results were surprising precisely because builders are not just base models. They’re base models plus scaffolding plus UX and post-processing.

My guidance for React developers:

• Use builders as idea generators. Harvest layout, copy, micro-interactions, then rebuild cleanly in your codebase

• Normalize APIs. Refactor generated fetch calls, hooks, stores to your patterns

• Consolidate CSS. Pull scattered styles into tokens and your component library to avoid spawning a second design system

• Archive failure cases. Save screenshots and diffs to refine prompts and tool settings over time

And if you want the “before you even start” checklist:

• Start with a written product spec: features, user types, flows

• Lock your design system: your existing shadcn, Radix, or in-house primitives

• Describe the vibe in concrete terms: reference sites, adjectives, motion levels

• Limit surface area: use builders for a single flow rather than your entire shell

If you treat builder output as production code by default, you’ll end up maintaining a foreign codebase you never chose.

6. UI Components Arena: where React developers win

The UI Components Arena is the most directly applicable to most React teams: generate isolated reusable components. Scope is focused, success rate is high, and output can be close to production-ready.

It’s also where the “logic not taste” lesson shows up cleanly: models can wire up props and state and still make ugly, inconsistent decisions.

So I use AI here heavily, but with a specific protocol.

Start by forcing a Component contract

These are the things I want in the prompt before the model writes JSX:

• Define prop names, types, variants, and states

• Provide examples and edge cases, including weird inputs

• Demand accessibility by default: keyboard nav, ARIA, focus management, error messaging

• Avoid anonymous div wrappers; use semantic structure where it matters

• Separate styling concerns: Tailwind classes or your utility system, not inline styles

• Request story files: Storybook or MDX usage examples

Then do the React integration work AI is usually bad at

Once you have a plausible component, integrate it like a senior engineer:

• Convert state to idiomatic React: Replace query selectors and global variables with hooks and props

• Make behavior composable: Refactor complex pieces into hooks you own

• Test the contract, not the implementation: Focus tests on props and events so internals can evolve

• Snap components into your design system before exposing them broadly

This is the pattern I recommend to teams: let AI get you 70% of the way on structure, then deliberately take ownership of API shape, composition, and design tokens.

Poor prompt:

Create a sign-up button component with different variants

Strong prompt:

Create a sign-up Button component with:

Props:
- variant: 'primary' | 'secondary' | 'ghost'
- size: 'sm' | 'md' | 'lg'
- disabled: boolean
- loading: boolean

Requirements:
- Use Tailwind classes
- Show loading spinner when loading=true
- Disable pointer events when disabled
- Support keyboard navigation (Enter/Space)
- Include focus-visible ring
- ARIA: use aria-disabled, aria-busy

Example usage:
<Button variant="primary" size="md" loading={isSubmitting}>
  Submit
</Button>

Then show a side-by-side of what each produces - the poor one generates inconsistent spacing, misses accessibility, uses inline styles. The strong one hits all requirements.

7. 3D and Data Viz: Let AI generate assets and data, not your entire integration

The 3D and Data Viz arenas stress more structured generation tasks, relevant for interactive dashboards, WebGL, and data-heavy apps.

The lesson from these arenas is not “AI writes your Three.js app.” It’s:

• Decide what AI should generate: Ask for geometries, datasets, configuration, not full integration code

• Specify the target library: React Three Fiber, Drei, Recharts, Victory, Visx

• Request low poly first and iterate toward fidelity once performance is proven

• Keep performance under control: Lazy load heavy assets, guard frame rate, keep fallbacks

In practice, this is how you avoid an “AI demo” becoming a performance incident.

React-Specific tips I want more teams to operationalize

The talk includes a slide of “React AI coding tips” that I keep coming back to because it captures what actually works in practice.

Here are the ones I see paying off immediately on real teams:

• Start prompts with the component API: Declare props, variants, states, and then tell the model to implement exactly that

• Name interactive states explicitly: hover, focus, loading, disabled

• Ask for a plan, then generate in steps: Small increments beat one big shot, and help avoid the complexity cliff

• Codify taste so AI can follow it: Lock spacing, colors, components in Tailwind config and your design system

• Be explicit about routes, layouts, server actions, loading and error boundaries

• Bake conventions into the repo: Document App Router defaults, Server Components, Suspense so assistants align automatically

• Run checks only on what changed: Husky with lint-staged to run typecheck, lint, tests on staged files

• Control cache behavior explicitly: Fetch cache options and revalidation windows as part of the prompt, so the model doesn’t guess your policy

The meta-message is the same: The difference between “AI helped me ship” and “AI gave me a mess” is almost always the level of specificity and the strength of your guardrails.

How I debug AI coding failures: it’s a pipeline, not a model

Once you accept the complexity cliff, the question becomes: how do you consistently get good outcomes?

I use a mental model I showed near the end of the talk: when AI code works or fails, it’s rarely “just the model.” It’s the whole pipeline.

The pipeline:

1. Base model

2. System prompt and instructions

3. Your user prompt

4. Fine-tuning and code training

5. Tools and retrieval (RAG)

6. Agent loops (iteration)

7. Post-processing

If you’re disappointed, you can almost always point to a weak link: wrong model for task, vague prompt, missing context, no iteration.

When things work, it’s usually because multiple layers aligned well: strong model, good prompt, necessary context, and iteration to iron out kinks.

This is actionable, because most of those layers are under your control as a user, even if you don’t own the base model.

The workflow I recommend for agentic React coding

I summarized it in the deck as the “new flow state”:

1. Define clear requirements (maybe write tests)

2. Prompt with context (stack, docs, examples)

3. Ask for plan, review

4. Generate code in small steps

5. Run, test, refine

6. Iterate until production-ready

This sounds like “normal engineering,” and that’s the point. The best teams I see using AI are not doing anything mystical. They’re turning implicit engineering discipline into explicit instructions and then using AI to accelerate the boring parts.

If you want one sentence: You’re not just typing code anymore, you’re orchestrating code creation.

So, how good is AI at coding React, really?

Where I land after looking at these benchmarks and using these tools day to day:

✅ AI is genuinely strong at:

• Isolated React components

• Scaffolding

• Converting clearly specified requirements into working code

⚠️ AI is still unreliable at:

• Multi-step integration tasks without strong tooling, strong context, and iteration loops

❌ AI is consistently weaker at:

• Taste, hierarchy, and nuanced UX decisions than it is at “code that runs”

• The aesthetic gap is real

💡 The highest leverage strategy is not “pick the best model.” It’s:

• Reduce context failures

• Codify your conventions

• Force stepwise work

And the part I find most exciting: The opportunity keeps expanding. Models and tools change fast, but the underlying skills that make you effective don’t. You are still the architect.

Learn More

If you want to keep exploring this space:

• Talk video: https://gitnation.com/contents/how-good-is-ai-at-coding-react-really

• Design Arena leaderboard: https://www.designarena.ai/leaderboard

• Design Arena methodology: https://notes.designarena.ai/in-pursuit-of-a-benchmark-for-human-taste/

And if you want to dive deeper into related topics, I’ve written two books on the topic: “Beyond Vibe Coding” and “Building large-scale web apps with React”

At Anthropic, for example, engineers adopted Claude Code so heavily that today ~90% of the code for Claude Code is written by Claude Code itself. Yet, using LLMs for programming is not a push-button magic experience - it’s “difficult and unintuitive” and getting great results requires learning new patterns. Critical thinking remains key. Over a year of projects, I’ve converged on a workflow similar to what many experienced devs are discovering: treat the LLM as a powerful pair programmer that requires clear direction, context and oversight rather than autonomous judgment.

In this article, I’ll share how I plan, code, and collaborate with AI going into 2026, distilling tips and best practices from my experience and the community’s collective learning. It’s a more disciplined “AI-assisted engineering” approach - leveraging AI aggressively while staying proudly accountable for the software produced.

If you’re interested in more on my workflow, see “The AI-Native Software Engineer”, otherwise let’s dive straight into some of the lessons I learned.

Start with a clear plan (specs before code)

Don’t just throw wishes at the LLM - begin by defining the problem and planning a solution.

One common mistake is diving straight into code generation with a vague prompt. In my workflow, and in many others’, the first step is brainstorming a detailed specification with the AI, then outlining a step-by-step plan, before writing any actual code. For a new project, I’ll describe the idea and ask the LLM to iteratively ask me questions until we’ve fleshed out requirements and edge cases. By the end, we compile this into a comprehensive spec.md - containing requirements, architecture decisions, data models, and even a testing strategy. This spec forms the foundation for development.

Next, I feed the spec into a reasoning-capable model and prompt it to generate a project plan: break the implementation into logical, bite-sized tasks or milestones. The AI essentially helps me do a mini “design doc” or project plan. I often iterate on this plan - editing and asking the AI to critique or refine it - until it’s coherent and complete. Only then do I proceed to coding. This upfront investment might feel slow, but it pays off enormously. As Les Orchard put it, it’s like doing a “waterfall in 15 minutes” - a rapid structured planning phase that makes the subsequent coding much smoother.

Having a clear spec and plan means when we unleash the codegen, both the human and the LLM know exactly what we’re building and why. In short, planning first forces you and the AI onto the same page and prevents wasted cycles. It’s a step many people are tempted to skip, but experienced LLM developers now treat a robust spec/plan as the cornerstone of the workflow.

Break work into small, iterative chunks

Scope management is everything - feed the LLM manageable tasks, not the whole codebase at once.

A crucial lesson I’ve learned is to avoid asking the AI for large, monolithic outputs. Instead, we break the project into iterative steps or tickets and tackle them one by one. This mirrors good software engineering practice, but it’s even more important with AI in the loop. LLMs do best when given focused prompts: implement one function, fix one bug, add one feature at a time. For example, after planning, I will prompt the codegen model: “Okay, let’s implement Step 1 from the plan”. We code that, test it, then move to Step 2, and so on. Each chunk is small enough that the AI can handle it within context and you can understand the code it produces.

This approach guards against the model going off the rails. If you ask for too much in one go, it’s likely to get confused or produce a “jumbled mess” that’s hard to untangle. Developers report that when they tried to have an LLM generate huge swaths of an app, they ended up with inconsistency and duplication - “like 10 devs worked on it without talking to each other,” one said. I’ve felt that pain; the fix is to stop, back up, and split the problem into smaller pieces. Each iteration, we carry forward the context of what’s been built and incrementally add to it. This also fits nicely with a test-driven development (TDD) approach - we can write or generate tests for each piece as we go (more on testing soon).

Several coding-agent tools now explicitly support this chunked workflow. For instance, I often generate a structured “prompt plan” file that contains a sequence of prompts for each task, so that tools like Cursor can execute them one by one. The key point is to avoid huge leaps. By iterating in small loops, we greatly reduce the chance of catastrophic errors and we can course-correct quickly. LLMs excel at quick, contained tasks - use that to your advantage.

Provide extensive context and guidance

LLMs are only as good as the context you provide - show them the relevant code, docs, and constraints.

When working on a codebase, I make sure to feed the AI all the information it needs to perform well. That includes the code it should modify or refer to, the project’s technical constraints, and any known pitfalls or preferred approaches. Modern tools help with this: for example, Anthropic’s Claude can import an entire GitHub repo into its context in “Projects” mode, and IDE assistants like Cursor or Copilot auto-include open files in the prompt. But I often go further - I will either use an MCP like Context7 or manually copy important pieces of the codebase or API docs into the conversation if I suspect the model doesn’t have them.

Expert LLM users emphasize this “context packing” step. For example, doing a “brain dump” of everything the model should know before coding, including: high-level goals and invariants, examples of good solutions, and warnings about approaches to avoid. If I’m asking an AI to implement a tricky solution, I might tell it which naive solutions are too slow, or provide a reference implementation from elsewhere. If I’m using a niche library or a brand-new API, I’ll paste in the official docs or README so the AI isn’t flying blind. All of this upfront context dramatically improves the quality of its output, because the model isn’t guessing - it has the facts and constraints in front of it.

There are now utilities to automate context packaging. I’ve experimented with tools like gitingest or repo2txt, which essentially “dump” the relevant parts of your codebase into a text file for the LLM to read. These can be a lifesaver when dealing with a large project - you generate an output.txt bundle of key source files and let the model ingest that. The principle is: don’t make the AI operate on partial information. If a bug fix requires understanding four different modules, show it those four modules. Yes, we must watch token limits, but current frontier models have pretty huge context windows (tens of thousands of tokens). Use them wisely. I often selectively include just the portions of code relevant to the task at hand, and explicitly tell the AI what not to focus on if something is out of scope (to save tokens).

I think Claude Skills have potential because they turn what used to be fragile repeated prompting into something durable and reusable by packaging instructions, scripts, and domain specific expertise into modular capabilities that tools can automatically apply when a request matches the Skill. This means you get more reliable and context aware results than a generic prompt ever could and you move away from one off interactions toward workflows that encode repeatable procedures and team knowledge for tasks in a consistent way. A number of community-curated Skills collections exist, but one of my favorite examples is the frontend-design skill which can “end” the purple design aesthetic prevalent in LLM generated UIs. Until more tools support Skills officially, workarounds exist.

Finally, guide the AI with comments and rules inside the prompt. I might precede a code snippet with: “Here is the current implementation of X. We need to extend it to do Y, but be careful not to break Z.” These little hints go a long way. LLMs are literalists - they’ll follow instructions, so give them detailed, contextual instructions. By proactively providing context and guidance, we minimize hallucinations and off-base suggestions and get code that fits our project’s needs.

Choose the right model (and use multiple when needed)

Not all coding LLMs are equal - pick your tool with intention, and don’t be afraid to swap models mid-stream.

In 2025 we’ve been spoiled with a variety of capable code-focused LLMs. Part of my workflow is choosing the model or service best suited to each task. Sometimes it can be valuable to even try two or more LLMs in parallel to cross-check how they might approach the same problem differently.

Each model has its own “personality”. The key is: if one model gets stuck or gives mediocre outputs, try another. I’ve literally copied the same prompt from one chat into another service to see if it can handle it better. This “model musical chairs” can rescue you when you hit a model’s blind spot.

Also, make sure you’re using the best version available. If you can, use the newest “pro” tier models - because quality matters. And yes, it often means paying for access, but the productivity gains can justify it. Ultimately, pick the AI pair programmer whose “vibe” meshes with you. I know folks who prefer one model simply because they like how its responses feel. That’s valid - when you’re essentially in a constant dialogue with an AI, the UX and tone make a difference.

Personally I gravitate towards Gemini for a lot of coding work these days because the interaction feels more natural and it often understands my requests on the first try. But I will not hesitate to switch to another model if needed; sometimes a second opinion helps the solution emerge. In summary: use the best tool for the job, and remember you have an arsenal of AIs at your disposal.

Leverage AI coding across the lifecycle

Supercharge your workflow with coding-specific AI help across the SDLC.

On the command-line, new AI agents emerged. Claude Code, OpenAI’s Codex CLI and Google’s Gemini CLI are CLI tools where you can chat with them directly in your project directory - they can read files, run tests, and even multi-step fix issues. I’ve used Google’s Jules and GitHub’s Copilot Agent as well - these are asynchronous coding agents that actually clone your repo into a cloud VM and work on tasks in the background (writing tests, fixing bugs, then opening a PR for you). It’s a bit eerie to witness: you issue a command like “refactor the payment module for X” and a little while later you get a pull request with code changes and passing tests. We are truly living in the future. You can read more about this in conductors to orchestrators.

That said, these tools are not infallible, and you must understand their limits. They accelerate the mechanical parts of coding - generating boilerplate, applying repetitive changes, running tests automatically - but they still benefit greatly from your guidance. For instance, when I use an agent like Claude or Copilot to implement something, I often supply it with the plan or to-do list from earlier steps so it knows the exact sequence of tasks. If the agent supports it, I’ll load up my spec.md or plan.md in the context before telling it to execute. This keeps it on track.

We’re not at the stage of letting an AI agent code an entire feature unattended and expecting perfect results. Instead, I use these tools in a supervised way: I’ll let them generate and even run code, but I keep an eye on each step, ready to step in when something looks off. There are also orchestration tools like Conductor that let you run multiple agents in parallel on different tasks (essentially a way to scale up AI help) - some engineers are experimenting with running 3-4 agents at once on separate features. I’ve dabbled in this “massively parallel” approach; it’s surprisingly effective at getting a lot done quickly, but it’s also mentally taxing to monitor multiple AI threads! For most cases, I stick to one main agent at a time and maybe a secondary one for reviews (discussed below).

Just remember these are power tools - you still control the trigger and guide the outcome.

A full overview of where AI can improve the developer experience. This spans design, inner, submit, and outer loops - highlighting every point where AI can meaningfully reduce toil.

Keep a human in the loop - verify, test, and review everything

AI will happily produce plausible-looking code, but you are responsible for quality - always review and test thoroughly. One of my cardinal rules is never to blindly trust an LLM’s output. As Simon Willison aptly says, think of an LLM pair programmer as “over-confident and prone to mistakes”. It writes code with complete conviction - including bugs or nonsense - and won’t tell you something is wrong unless you catch it. So I treat every AI-generated snippet as if it came from a junior developer: I read through the code, run it, and test it as needed. You absolutely have to test what it writes - run those unit tests, or manually exercise the feature, to ensure it does what it claims. Read more about this in vibe coding is not an excuse for low-quality work.

In fact, I weave testing into the workflow itself. My earlier planning stage often includes generating a list of tests or a testing plan for each step. If I’m using a tool like Claude Code, I’ll instruct it to run the test suite after implementing a task, and have it debug failures if any occur. This kind of tight feedback loop (write code → run tests → fix) is something AI excels at as long as the tests exist. It’s no surprise that those who get the most out of coding agents tend to be those with strong testing practices. An agent like Claude can “fly” through a project with a good test suite as safety net. Without tests, the agent might blithely assume everything is fine (“sure, all good!”) when in reality it’s broken several things. So, invest in tests - it amplifies the AI’s usefulness and confidence in the result.

Even beyond automated tests, do code reviews - both manual and AI-assisted. I routinely pause and review the code that’s been generated so far, line by line. Sometimes I’ll spawn a second AI session (or a different model) and ask it to critique or review code produced by the first. For example, I might have Claude write the code and then ask Gemini, “Can you review this function for any errors or improvements?” This can catch subtle issues. The key is to not skip the review just because an AI wrote the code. If anything, AI-written code needs extra scrutiny, because it can sometimes be superficially convincing while hiding flaws that a human might not immediately notice.

I also use Chrome DevTools MCP, built with my last team, for my debugging and quality loop to bridge the gap between static code analysis and live browser execution. It “gives your agent eyes”. It lets me grant my AI tools direct access to see what the browser can, inspect the DOM, get rich performance traces, console logs or network traces. This integration eliminates the friction of manual context switching, allowing for automated UI testing directly through the LLM. It means bugs can be diagnosed and fixed with high precision based on actual runtime data.

The dire consequences of skipping human oversight have been documented. One developer who leaned heavily on AI generation for a rush project described the result as an inconsistent mess - duplicate logic, mismatched method names, no coherent architecture. He realized he’d been “building, building, building” without stepping back to really see what the AI had woven together. The fix was a painful refactor and a vow to never let things get that far out of hand again. I’ve taken that to heart. No matter how much AI I use, I remain the accountable engineer.

In practical terms, that means I only merge or ship code after I’ve understood it. If the AI generates something convoluted, I’ll ask it to add comments explaining it, or I’ll rewrite it in simpler terms. If something doesn’t feel right, I dig in - just as I would if a human colleague contributed code that raised red flags.

It’s all about mindset: the LLM is an assistant, not an autonomously reliable coder. I am the senior dev; the LLM is there to accelerate me, not replace my judgment. Maintaining this stance not only results in better code, it also protects your own growth as a developer. (I’ve heard some express concern that relying too much on AI might dull their skills - I think as long as you stay in the loop, actively reviewing and understanding everything, you’re still sharpening your instincts, just at a higher velocity.) In short: stay alert, test often, review always. It’s still your codebase at the end of the day.

Commit often and use version control as a safety net. Never commit code you can’t explain.

Frequent commits are your save points - they let you undo AI missteps and understand changes.

When working with an AI that can generate a lot of code quickly, it’s easy for things to veer off course. I mitigate this by adopting ultra-granular version control habits. I commit early and often, even more than I would in normal hand-coding. After each small task or each successful automated edit, I’ll make a git commit with a clear message. This way, if the AI’s next suggestion introduces a bug or a messy change, I have a recent checkpoint to revert to (or cherry-pick from) without losing hours of work. One practitioner likened it to treating commits as “save points in a game” - if an LLM session goes sideways, you can always roll back to the last stable commit. I’ve found that advice incredibly useful. It’s much less stressful to experiment with a bold AI refactor when you know you can undo it with a git reset if needed.

Proper version control also helps when collaborating with the AI. Since I can’t rely on the AI to remember everything it’s done (context window limitations, etc.), the git history becomes a valuable log. I often scan my recent commits to brief the AI (or myself) on what changed. In fact, LLMs themselves can leverage your commit history if you provide it - I’ve pasted git diffs or commit logs into the prompt so the AI knows what code is new or what the previous state was. Amusingly, LLMs are really good at parsing diffs and using tools like git bisect to find where a bug was introduced. They have infinite patience to traverse commit histories, which can augment your debugging. But this only works if you have a tidy commit history to begin with.

Another benefit: small commits with good messages essentially document the development process, which helps when doing code review (AI or human). If an AI agent made five changes in one go and something broke, having those changes in separate commits makes it easier to pinpoint which commit caused the issue. If everything is in one giant commit titled “AI changes”, good luck! So I discipline myself: finish task, run tests, commit. This also meshes well with the earlier tip about breaking work into small chunks - each chunk ends up as its own commit or PR.

Finally, don’t be afraid to use branches or worktrees to isolate AI experiments. One advanced workflow I’ve adopted (inspired by folks like Jesse Vincent) is to spin up a fresh git worktree for a new feature or sub-project. This lets me run multiple AI coding sessions in parallel on the same repo without them interfering, and I can later merge the changes. It’s a bit like having each AI task in its own sandbox branch. If one experiment fails, I throw away that worktree and nothing is lost in main. If it succeeds, I merge it in. This approach has been crucial when I’m, say, letting an AI implement Feature A while I (or another AI) work on Feature B simultaneously. Version control is what makes this coordination possible. In short: commit often, organize your work with branches, and embrace git as the control mechanism to keep AI-generated changes manageable and reversible.

Customize the AI’s behavior with rules and examples

Steer your AI assistant by providing style guides, examples, and even “rules files” - a little upfront tuning yields much better outputs.

One thing I learned is that you don’t have to accept the AI’s default style or approach - you can influence it heavily by giving it guidelines. For instance, I have a CLAUDE.md file that I update periodically, which contains process rules and preferences for Claude (Anthropic’s model) to follow (and similarly a GEMINI.md when using Gemini CLI). This includes things like “write code in our project’s style, follow our lint rules, don’t use certain functions, prefer functional style over OOP,” etc. When I start a session, I feed this file to Claude to align it with our conventions. It’s surprising how well this works to keep the model “on track” as Jesse Vincent noted - it reduces the tendency of the AI to go off-script or introduce patterns we don’t want.

Even without a fancy rules file, you can set the tone with custom instructions or system prompts. GitHub Copilot and Cursor both introduced features to let you configure the AI’s behavior globally for your project. I’ve taken advantage of that by writing a short paragraph about our coding style, e.g. “Use 4 spaces indent, avoid arrow functions in React, prefer descriptive variable names, code should pass ESLint.” With those instructions in place, the AI’s suggestions adhere much more closely to what a human teammate might write. Ben Congdon mentioned how shocked he was that few people use Copilot’s custom instructions, given how effective they are - he could guide the AI to output code matching his team’s idioms by providing some examples and preferences upfront. I echo that: take the time to teach the AI your expectations.

Another powerful technique is providing in-line examples of the output format or approach you want. If I want the AI to write a function in a very specific way, I might first show it a similar function already in the codebase: “Here’s how we implemented X, use a similar approach for Y.” If I want a certain commenting style, I might write a comment myself and ask the AI to continue in that style. Essentially, prime the model with the pattern to follow. LLMs are great at mimicry - show them one or two examples and they’ll continue in that vein.

The community has also come up with creative “rulesets” to tame LLM behavior. You might have heard of the “Big Daddy” rule or adding a “no hallucination/no deception” clause to prompts. These are basically tricks to remind the AI to be truthful and not overly fabricate code that doesn’t exist. For example, I sometimes prepend a prompt with: “If you are unsure about something or the codebase context is missing, ask for clarification rather than making up an answer.” This reduces hallucinations. Another rule I use is: “Always explain your reasoning briefly in comments when fixing a bug.” This way, when the AI generates a fix, it will also leave a comment like “// Fixed: Changed X to Y to prevent Z (as per spec).” That’s super useful for later review.

In summary, don’t treat the AI as a black box - tune it. By configuring system instructions, sharing project docs, or writing down explicit rules, you turn the AI into a more specialized developer on your team. It’s akin to onboarding a new hire: you’d give them the style guide and some starter tips, right? Do the same for your AI pair programmer. The return on investment is huge: you get outputs that need less tweaking and integrate more smoothly with your codebase.

Embrace testing and automation as force multipliers

Use your CI/CD, linters, and code review bots - AI will work best in an environment that catches mistakes automatically.

This is a corollary to staying in the loop and providing context: a well-oiled development pipeline enhances AI productivity. I ensure that any repository where I use heavy AI coding has a robust continuous integration setup. That means automated tests run on every commit or PR, code style checks (like ESLint, Prettier, etc.) are enforced, and ideally a staging deployment is available for any new branch. Why? Because I can let the AI trigger these and evaluate the results. For instance, if the AI opens a pull request via a tool like Jules or GitHub Copilot Agent, our CI will run tests and report failures. I can feed those failure logs back to the AI: “The integration tests failed with XYZ, let’s debug this.” It turns bug-fixing into a collaborative loop with quick feedback, which AIs handle quite well (they’ll suggest a fix, we run CI again, and iterate).

Automated code quality checks (linters, type checkers) also guide the AI. I actually include linter output in the prompt sometimes. If the AI writes code that doesn’t pass our linter, I’ll copy the linter errors into the chat and say “please address these issues.” The model then knows exactly what to do. It’s like having a strict teacher looking over the AI’s shoulder. In my experience, once the AI is aware of a tool’s output (like a failing test or a lint warning), it will try very hard to correct it - after all, it “wants” to produce the right answer. This ties back to providing context: give the AI the results of its actions in the environment (test failures, etc.) and it will learn from them.

AI coding agents themselves are increasingly incorporating automation hooks. Some agents will refuse to say a code task is “done” until all tests pass, which is exactly the diligence you want. Code review bots (AI or otherwise) act as another filter - I treat their feedback as additional prompts for improvement. For example, if CodeRabbit or another reviewer comments “This function is doing X which is not ideal” I will ask the AI, “Can you refactor based on this feedback?”

By combining AI with automation, you start to get a virtuous cycle. The AI writes code, the automated tools catch issues, the AI fixes them, and so forth, with you overseeing the high-level direction. It feels like having an extremely fast junior dev whose work is instantly checked by a tireless QA engineer. But remember, you set up that environment. If your project lacks tests or any automated checks, the AI’s work may slip through with subtle bugs or poor quality until much later.

So as we head into 2026, one of my goals is to bolster the quality gates around AI code contribution: more tests, more monitoring, perhaps even AI-on-AI code reviews. It might sound paradoxical (AIs reviewing AIs), but I’ve seen it catch things one model missed. Bottom line: an AI-friendly workflow is one with strong automation - use those tools to keep the AI honest.

Continuously learn and adapt (AI amplifies your skills)

Treat every AI coding session as a learning opportunity - the more you know, the more the AI can help you, creating a virtuous cycle.

One of the most exciting aspects of using LLMs in development is how much I have learned in the process. Rather than replacing my need to know things, AIs have actually exposed me to new languages, frameworks, and techniques I might not have tried on my own.

This pattern holds generally: if you come to the table with solid software engineering fundamentals, the AI will amplify your productivity multifold. If you lack that foundation, the AI might just amplify confusion. Seasoned devs have observed that LLMs “reward existing best practices” - things like writing clear specs, having good tests, doing code reviews, etc., all become even more powerful when an AI is involved. In my experience, the AI lets me operate at a higher level of abstraction (focusing on design, interface, architecture) while it churns out the boilerplate, but I need to have those high-level skills first. As Simon Willison notes, almost everything that makes someone a senior engineer (designing systems, managing complexity, knowing what to automate vs hand-code) is what now yields the best outcomes with AI. So using AIs has actually pushed me to up my engineering game - I’m more rigorous about planning and more conscious of architecture, because I’m effectively “managing” a very fast but somewhat naïve coder (the AI).

For those worried that using AI might degrade their abilities: I’d argue the opposite, if done right. By reviewing AI code, I’ve been exposed to new idioms and solutions. By debugging AI mistakes, I’ve deepened my understanding of the language and problem domain. I often ask the AI to explain its code or the rationale behind a fix - kind of like constantly interviewing a candidate about their code - and I pick up insights from its answers. I also use AI as a research assistant: if I’m not sure about a library or approach, I’ll ask it to enumerate options or compare trade-offs. It’s like having an encyclopedic mentor on call. All of this has made me a more knowledgeable programmer.

The big picture is that AI tools amplify your expertise. Going into 2026, I’m not afraid of them “taking my job” - I’m excited that they free me from drudgery and allow me to spend more time on creative and complex aspects of software engineering. But I’m also aware that for those without a solid base, AI can lead to Dunning-Kruger on steroids (it may seem like you built something great, until it falls apart). So my advice: continue honing your craft, and use the AI to accelerate that process. Be intentional about periodically coding without AI too, to keep your raw skills sharp. In the end, the developer + AI duo is far more powerful than either alone, and the developer half of that duo has to hold up their end.

Conclusion

As we enter 2026, I’ve fully embraced AI in my development workflow - but in a considered, expert-driven way. My approach is essentially “AI-augmented software engineering” rather than AI-automated software engineering.

I’ve learned: the best results come when you apply classic software engineering discipline to your AI collaborations. It turns out all our hard-earned practices - design before coding, write tests, use version control, maintain standards - not only still apply, but are even more important when an AI is writing half your code.

I’m excited for what’s next. The tools keep improving and my workflow will surely evolve alongside them. Perhaps fully autonomous “AI dev interns” will tackle more grunt work while we focus on higher-level tasks. Perhaps new paradigms of debugging and code exploration will emerge. No matter what, I plan to stay in the loop - guiding the AIs, learning from them, and amplifying my productivity responsibly.

The bottom line for me: AI coding assistants are incredible force multipliers, but the human engineer remains the director of the show.

With that…happy building in 2026! 🚀

I’m excited to share I’ve released a new AI-assisted engineering book with O’Reilly. There are a number of free tips on the book site in case interested.

These lessons are what I wish I’d known earlier. Some would have saved me months of frustration. Others took years to fully understand. None of them are about specific technologies - those change too fast to matter. They’re about the patterns that keep showing up, project after project, team after team.

I’m sharing them because I’ve benefited enormously from engineers who did the same for me. Consider this my attempt to pay it forward.

1. The best engineers are obsessed with solving user problems.

It’s seductive to fall in love with a technology and go looking for places to apply it. I’ve done it. Everyone has. But the engineers who create the most value work backwards: they become obsessed with understanding user problems deeply, and let solutions emerge from that understanding.

User obsession means spending time in support tickets, talking to users, watching users struggle, asking “why” until you hit bedrock. The engineer who truly understands the problem often finds that the elegant solution is simpler than anyone expected.

The engineer who starts with a solution tends to build complexity in search of a justification.

2. Being right is cheap. Getting to right together is the real work.

You can win every technical argument and lose the project. I’ve watched brilliant engineers accrue silent resentment by always being the smartest person in the room. The cost shows up later as “mysterious execution issues” and “strange resistance.”

The skill isn’t being right. It’s entering discussions to align on the problem, creating space for others, and remaining skeptical of your own certainty.

Strong opinions, weakly held - not because you lack conviction, but because decisions made under uncertainty shouldn’t be welded to identity.

3. Bias towards action. Ship. You can edit a bad page, but you can’t edit a blank one.

The quest for perfection is paralyzing. I’ve watched engineers spend weeks debating the ideal architecture for something they’ve never built. The perfect solution rarely emerges from thought alone - it emerges from contact with reality. AI can in many ways help here.

First do it, then do it right, then do it better. Get the ugly prototype in front of users. Write the messy first draft of the design doc. Ship the MVP that embarrasses you slightly. You’ll learn more from one week of real feedback than a month of theoretical debate.

Momentum creates clarity. Analysis paralysis creates nothing.

4. Clarity is seniority. Cleverness is overhead.

The instinct to write clever code is almost universal among engineers. It feels like proof of competence.

But software engineering is what happens when you add time and other programmers. In that environment, clarity isn’t a style preference - it’s operational risk reduction.

Your code is a strategy memo to strangers who will maintain it at 2am during an outage. Optimize for their comprehension, not your elegance. The senior engineers I respect most have learned to trade cleverness for clarity, every time.

5. Novelty is a loan you repay in outages, hiring, and cognitive overhead.

Treat your technology choices like an organization with a small “innovation token” budget. Spend one each time you adopt something materially non-standard. You can’t afford many.

The punchline isn’t “never innovate.” It’s “innovate only where you’re uniquely paid to innovate.” Everything else should default to boring, because boring has known failure modes.

The “best tool for the job” is often the “least-worst tool across many jobs”-because operating a zoo becomes the real tax.

6. Your code doesn’t advocate for you. People do.

Early in my career, I believed great work would speak for itself. I was wrong. Code sits silently in a repository. Your manager mentions you in a meeting, or they don’t. A peer recommends you for a project, or someone else.

In large organizations, decisions get made in meetings you’re not invited to, using summaries you didn’t write, by people who have five minutes and twelve priorities. If no one can articulate your impact when you’re not in the room, your impact is effectively optional.

This isn’t strictly about self-promotion. It’s about making the value chain legible to everyone- including yourself.

7. The best code is the code you never had to write.

We celebrate creation in engineering culture. Nobody gets promoted for deleting code, even though deletion often improves a system more than addition. Every line of code you don’t write is a line you never have to debug, maintain, or explain.

Before you build, exhaust the question: “What would happen if we just… didn’t?” Sometimes the answer is “nothing bad,” and that’s your solution.

The problem isn’t that engineers can’t write code or use AI to do so. It’s that we’re so good at writing it that we forget to ask whether we should.

8. At scale, even your bugs have users.

With enough users, every observable behavior becomes a dependency - regardless of what you promised. Someone is scraping your API, automating your quirks, caching your bugs.

This creates a career-level insight: you can’t treat compatibility work as “maintenance” and new features as “real work.” Compatibility is product.

Design your deprecations as migrations with time, tooling, and empathy. Most “API design” is actually “API retirement.”

9. Most “slow” teams are actually misaligned teams.

When a project drags, the instinct is to blame execution: people aren’t working hard enough, the technology is wrong, there aren’t enough engineers. Usually none of that is the real problem.

In large companies, teams are your unit of concurrency, but coordination costs grow geometrically as teams multiply. Most slowness is actually alignment failure - people building the wrong things, or the right things in incompatible ways.

Senior engineers spend more time clarifying direction, interfaces, and priorities than “writing code faster” because that’s where the actual bottleneck lives.

10. Focus on what you can control. Ignore what you can’t.

In a large company, countless variables are outside your control - organizational changes, management decisions, market shifts, product pivots. Dwelling on these creates anxiety without agency.

The engineers who stay sane and effective zero in on their sphere of influence. You can’t control whether a reorg happens. You can control the quality of your work, how you respond, and what you learn. When faced with uncertainty, break problems into pieces and identify the specific actions available to you.

This isn’t passive acceptance but it is strategic focus. Energy spent on what you can’t change is energy stolen from what you can.

11. Abstractions don’t remove complexity. They move it to the day you’re on call.

Every abstraction is a bet that you won’t need to understand what’s underneath. Sometimes you win that bet. But something always leaks, and when it does, you need to know what you’re standing on.

Senior engineers keep learning “lower level” things even as stacks get higher. Not out of nostalgia, but out of respect for the moment when the abstraction fails and you’re alone with the system at 3am. Use your stack.

But keep a working model of its underlying failure modes.

12. Writing forces clarity. The fastest way to learn something better is to try teaching it.

Writing forces clarity. When I explain a concept to others - in a doc, a talk, a code review comment, even just chatting with AI - I discover the gaps in my own understanding. The act of making something legible to someone else makes it more legible to me.

This doesn’t mean that you’re going to learn how to be a surgeon by teaching it, but the premise still holds largely true in the software engineering domain.

This isn’t just about being generous with knowledge. It’s a selfish learning hack. If you think you understand something, try to explain it simply. The places where you stumble are the places where your understanding is shallow.

Teaching is debugging your own mental models.

13. The work that makes other work possible is priceless - and invisible.

Glue work - documentation, onboarding, cross-team coordination, process improvement - is vital. But if you do it unconsciously, it can stall your technical trajectory and burn you out. The trap is doing it as “helpfulness” rather than treating it as deliberate, bounded, visible impact.

Timebox it. Rotate it. Turn it into artifacts: docs, templates, automation. And make it legible as impact, not as personality trait.

Priceless and invisible is a dangerous combination for your career.

14. If you win every debate, you’re probably accumulating silent resistance.

I’ve learned to be suspicious of my own certainty. When I “win” too easily, something is usually wrong. People stop fighting you not because you’ve convinced them, but because they’ve given up trying - and they’ll express that disagreement in execution, not meetings.

Real alignment takes longer. You have to actually understand other perspectives, incorporate feedback, and sometimes change your mind publicly.

The short-term feeling of being right is worth much less than the long-term reality of building things with willing collaborators.

15. When a measure becomes a target, it stops measuring.

Every metric you expose to management will eventually be gamed. Not through malice, but because humans optimize for what’s measured.

If you track lines of code, you’ll get more lines. If you track velocity, you’ll get inflated estimates.

The senior move: respond to every metric request with a pair. One for speed. One for quality or risk. Then insist on interpreting trends, not worshiping thresholds. The goal is insight, not surveillance.

16. Admitting what you don’t know creates more safety than pretending you do.

Senior engineers who say “I don’t know” aren’t showing weakness - they’re creating permission. When a leader admits uncertainty, it signals that the room is safe for others to do the same. The alternative is a culture where everyone pretends to understand and problems stay hidden until they explode.

I’ve seen teams where the most senior person never admitted confusion, and I’ve seen the damage. Questions don’t get asked. Assumptions don’t get challenged. Junior engineers stay silent because they assume everyone else gets it.

Model curiosity, and you get a team that actually learns.

17. Your network outlasts every job you’ll ever have.

Early in my career, I focused on the work and neglected networking. In hindsight, this was a mistake. Colleagues who invested in relationships - inside and outside the company - reaped benefits for decades.

They heard about opportunities first, could build bridges faster, got recommended for roles, and co-founded ventures with people they’d built trust with over years.

Your job isn’t forever, but your network is. Approach it with curiosity and generosity, not transactional hustle.

When the time comes to move on, it’s often relationships that open the door.

18. Most performance wins come from removing work, not adding cleverness.

When systems get slow, the instinct is to add: caching layers, parallel processing, smarter algorithms. Sometimes that’s right. But I’ve seen more performance wins from asking “what are we computing that we don’t need?”

Deleting unnecessary work is almost always more impactful than doing necessary work faster. The fastest code is code that never runs.

Before you optimize, question whether the work should exist at all.

19. Process exists to reduce uncertainty, not to create paper trails.

The best process makes coordination easier and failures cheaper. The worst process is bureaucratic theater - it exists not to help but to assign blame when things go wrong.

If you can’t explain how a process reduces risk or increases clarity, it’s probably just overhead.

And if people are spending more time documenting their work than doing it, something has gone deeply wrong.

20. Eventually, time becomes worth more than money. Act accordingly.

Early in your career, you trade time for money - and that’s fine. But at some point, the calculus inverts. You start to realize that time is the non-renewable resource.

I’ve watched senior engineers burn out chasing the next promo level, optimizing for a few more percentage points of compensation. Some of them got it. Most of them wondered, afterward, if it was worth what they gave up.

The answer isn’t “don’t work hard.” It’s “know what you’re trading, and make the trade deliberately.”

21. There are no shortcuts, but there is compounding.

Expertise comes from deliberate practice - pushing slightly beyond your current skill, reflecting, repeating. For years. There’s no condensed version.

But here’s the hopeful part: learning compounds when it creates new options, not just new trivia. Write - not for engagement, but for clarity. Build reusable primitives. Collect scar tissue into playbooks.

The engineer who treats their career as compound interest, not lottery tickets, tends to end up much further ahead.

A final thought

Twenty-one lessons sounds like a lot, but they really come down to a few core ideas: stay curious, stay humble, and remember that the work is always about people - the users you’re building for and the teammates you’re building with.

A career in engineering is long enough to make plenty of mistakes and still come out ahead. The engineers I admire most aren’t the ones who got everything right - they’re the ones who learned from what went wrong, shared what they discovered, and kept showing up.

If you’re early in your journey, know that it gets richer with time. If you’re deep into it, I hope some of these resonate.