I’ll tell you, Lars, what’s wrong:

“Waterfall was coined in the 70’s as a type of project structure for software development.”

Wrong twice over.

Nobody coined it as a methodology. There is no founding paper, no author, no manifesto, no standards body that sat down and said “let us call this Waterfall.” The word was applied retrospectively, by other people, mostly as criticism. Its first known appearance in print is Bell and Thayer’s 1976 ICSE paper Software Requirements: Are They Really A Problem?, and they were describing someone else’s diagram.

That someone else was Winston Royce, 1970, Managing the Development of Large Software Systems. He never used the word. More importantly, he drew the sequential model in Figure 2 and then wrote, in the very next breath, that the implementation as described “is risky and invites failure.” The remaining two thirds of the paper is five recommended fixes, one of which is literally “do it twice”, a prototype iteration, and another of which is “involve the customer.” Royce is cited as the father of waterfall by people who read one diagram and stopped.

And the sequential phase model predates Royce by fourteen years anyway. Herbert Benington described it in 1956, presenting how SAGE was built. In his 1983 retrospective he added the detail everyone omits: they built a prototype first.

Waterfall as an actual mandated practice comes from procurement, not engineering. DOD-STD-2167 in 1985 effectively baked it into US defence contracting. MIL-STD-498 unbaked it in 1994 by explicitly permitting evolutionary and incremental lifecycles. So the one institution that genuinely did codify waterfall spent nine years doing it and then stopped.

“Non-software engineering fields use traditional sequential frameworks that mirror the waterfall model.”

This is the load-bearing claim and it is not true.

Naval architects have used the design spiral since at least Evans in 1959. It is called a spiral because you go round it repeatedly. Aerospace conceptual design runs iterative sizing loops. Toyota’s product development uses set-based concurrent engineering, deliberately carrying multiple design alternatives forward rather than committing early, documented by Ward, Liker and Sobek in The Second Toyota Paradox. Construction has the Last Planner System and Integrated Project Delivery, and lean construction has been an organised movement since 1997. SpaceX’s entire hardware method is build, fly, explode, learn, repeat.

Systems engineering does not use waterfall either. It uses the V-model, which is not a sequence but a decomposition paired with verification and validation looping back to each corresponding level. ISO/IEC/IEEE 15288 and the INCOSE handbook both explicitly enumerate incremental and evolutionary lifecycle models as legitimate.

What is true is that construction execution is sequential, because you cannot roof a building before you wall it. That is physics, not project management philosophy. The design phase preceding it is iterative as hell.

“Only software development engineering frown upon it.”

See above. Lean construction, lean product development, set-based design and the Toyota Production System are all pushbacks on sequential single-pass design, and none of them came out of software.

The unexamined question: why do other disciplines front-load design?

Because their cost of change curve is brutal. Moving a load-bearing wall after the concrete cures costs a fortune. Recalling a shipped airframe costs more. When change is expensive, you buy insurance by thinking harder up front, and that is rational.

Software’s marginal cost of change is close to zero, and modern engineering practice, continuous delivery, automated testing, trunk based development, is a deliberate campaign to keep it there. Copying the process of a discipline whose economics are the opposite of yours is cargo culting. The post treats the sequence as the principle. The sequence is a consequence.

“Fixed price, fixed date. I asked an agile team when they’d be finished and they said we don’t know.”

They were a bad team. You have described one anecdote and generalised it to a discipline.

Empirical forecasting is not exotic. Throughput, cycle time distributions and Monte Carlo simulation over historical data will give you a date with a confidence interval, which is more than a Gantt chart has ever given anyone. Vacanti’s Actionable Agile Metrics for Predictability is the standard reference. A plan produces a confident date. Empirical forecasting produces an accurate one. These are different things and the industry has spent fifty years confusing them.

Also, fixed price and fixed date do not make an estimate correct. They transfer risk to the supplier, who prices the risk in and then manages scope quietly. Everyone flexes scope. Agile flexes it visibly and early. Waterfall flexes it in month eleven of a twelve month contract.

And fixed price agile contracts exist and have for two decades: the Norwegian PS2000 model, Sutherland’s “money for nothing, change for free”, capped time and materials with a fixed scope reserve.

“Traditional engineering is better at hitting dates.”

Flyvbjerg’s dataset of over 16,000 projects says otherwise. Roughly 47.9% come in on budget. About 8.5% hit budget and schedule. Around 0.5% hit budget, schedule and benefits. The exemplars of the sequential framework include the F-35, the 787, Berlin Brandenburg Airport, Crossrail and HS2. If you want to hold up traditional engineering as the predictability benchmark, you should look at the data first.

Note also what Flyvbjerg actually recommends: modularity, repeatable units, and slow deliberate planning that is itself iterative, the Pixar model of planning by rewriting. That is not waterfall.

“Agile is a software development framework, it was intended for software development.”

Historically backwards. Scrum takes its name and its central idea from Takeuchi and Nonaka’s 1986 HBR article The New New Product Development Game, a study of overlapping-phase product development at Fuji-Xerox, Canon, Honda, NEC and Epson. Copiers, cameras, cars and printers. Scrum was imported into software from manufacturing, not exported out of it.

Kanban is Toyota. Lean is Toyota. PDCA is Shewhart and Deming, from statistical process control in manufacturing. Software borrowed nearly all of it. Arguing that agile cannot leave software is arguing that it cannot go home.

This is also just the genetic fallacy. Where an idea originated tells you nothing about where it applies.

“Worldwide router upgrade, package upgrade, cloud migration. NO.”

Here Lars is half right, for entirely the wrong reason.

The variable that matters is not software versus not-software. It is how much uncertainty the work contains. Snowden’s Cynefin framework is the useful lens: work that is clear or complicated has knowable answers and expert practice, so plan it and execute the plan. Work that is complex has answers that only emerge through probing, so run experiments and adapt.

A worldwide router upgrade is complicated. Plan it. Fine. Nobody serious is proposing a daily standup to discuss whether the router should exist.

But cloud migrations fail precisely because people classify them as complicated when they are complex. The undocumented dependency, the licence that does not port, the service nobody knew was still in the critical path. Which is why competent migrations run in waves, with a pilot, learning from each wave. That is incremental delivery with feedback. You can refuse to call it agile if the word upsets you, but that is what it is.

And note the third category you have skipped: your example of “upgrading a software package to a new version” being unsuitable for agile is odd, given that shipping small, frequent, reversible changes is exactly what DORA’s decade of research associates with both higher throughput and higher stability. Big-bang version upgrades are the failure mode, not the safe option.

Where this actually leaves us

The post is arguing about two labels when the real variable is the cost of being wrong divided by the speed at which you find out. Front-load your thinking when discovering you were wrong is expensive and slow. Shorten your feedback loops when it is cheap and fast. Most real work sits between the two, and the sensible answer is a plan you fully intend to revise, which is neither of the two tribes the post is asking you to pick.

Waterfall is not a framework anyone designed. It is a shape people noticed in a diagram drawn by a man who was warning them about it.

Most “AI coding” setups are a single model doing everything: it talks to you, designs the solution, writes the code, and tells you it’s done. That works until it doesn’t — and when it doesn’t, you can’t see where the reasoning went wrong, because it all happened inside one opaque conversation.

The Relay Method takes the opposite stance. It splits the work across a small chain of specialized agents, each with one job and one rule: you may only talk to the agent on your left and the agent on your right. A human problem flows down the chain, getting sharper at every step; working software and status flow back up. Every message between agents is a file on disk, appended to a single audit log you can replay line by line.

The result is a system where each handoff is a translation between levels of abstraction, where implementation detail physically cannot leak up to the human and the human’s framing cannot leak down into the code, and where — months later — you can reconstruct exactly how “I want a 3D Tetris game” became a specific commit.

This article describes the method, each persona in the chain, how they communicate, and why the constraints are the point.

The core idea: a bucket brigade for abstraction

Picture five stations in a line:

Owner (human)  ⇄  Interpreter  ⇄  Analyst  ⇄  Examiner  ⇄  Builder
└── live chat ──┘  └──────────── relay messages ────────────┘

Each arrow is an edge, and every edge is a translation to a different level of abstraction. Going down the chain, the same intent is progressively sharpened:

• The Owner has a problem, in plain human words.

• The Interpreter restates it as a need to implement — still in the Owner’s language, never in solution terms.

• The Analyst reframes that need as an observable behaviour — what must be true, with no hint of how.

• The Examiner decomposes the behaviour into expectations — precise, checkable statements.

• The Builder writes code and produces evidence that each expectation holds.

The neighbours-only rule is not bureaucracy; it is the mechanism. Because the Builder can only speak to the Examiner, it can never leak file names and data structures up to the human. Because the Interpreter can only speak to the Analyst (downward) and the Owner (upward), it can never smuggle business framing into the code. Each role is forced to speak the vocabulary of its own edge — and that is exactly what keeps the abstraction levels clean.

There’s a sixth participant who stands outside the line — the Sentinel — whose job is to make sure everyone honours their edge. More on them later.

The personas

1. The Owner — the human with a problem

The Owner is you. You don’t write code, you don’t design the architecture, and you don’t manage the agents. You state a problem, answer clarifying questions, approve a plan, and then gate each increment: continue, stop, or change course.

This is the quiet revolution of the method: your role shifts from author to editor. You are no longer producing the solution; you are evaluating proposals and steering. You speak to exactly one agent — the Interpreter — and only ever in the language of needs and outcomes.

2. The Interpreter — the Owner’s voice inside the machine

The Interpreter is the bilingual diplomat. Upward, it speaks human: it asks you clarifying questions before planning (so ambiguity is resolved rather than assumed), proposes a roadmap of potentially-shippable iterations, presents finished increments, and asks whether to keep going. Downward, it hands the Analyst one behaviour to implement at a time — still expressed as a need (”the player should see the piece fall and lock at the bottom”), never as a design.

Crucially, the Interpreter is the only edge that is a live conversation: you type in its window and it talks back. Everything below it happens through files, asynchronously, while you watch.

3. The Analyst — from need to observable behaviour

The Analyst is the translator that strips solutions out of needs. It receives a behaviour-to-implement and reframes it as a pure behaviour: an actor, an observable outcome, and explicit boundaries — with every trace of how removed.

“Build a falling piece” becomes: “Without any user interaction, the piece descends by exactly one cell at a steady, observable cadence; it stays aligned to the grid at each step; this covers descent through empty space only — what happens at the floor is a separate behaviour.”

Notice what’s absent: no mention of timers, frames, data structures, or languages. The Analyst answers what must be observably true, and hands that to the Examiner. When status comes back up, the Analyst translates it the other way — reporting to the Interpreter only which problem was solved, never expectations or tests.

4. The Examiner — expectations, not assertions

The Examiner is where the method gets its rigor, and it runs on Expectation-Driven Development (EDD) — a practice for human–AI collaboration in which correctness is established by stating expectations in plain language and then demanding evidence that they hold, rather than by anyone declaring “done.” (See the original write-up: Expectation-Driven Development.)

The Examiner takes a behaviour and decomposes it into a set of expectations — E1..En — each a precise, checkable statement of what must be true, plus an end-to-end integration expectation that ties them together. For the falling piece: “E1: the piece’s vertical position decreases by exactly one cell per tick. E2: between ticks it is stationary. E3: it never leaves the grid…” Still no “how” — expectations describe outcomes, not mechanisms.

It sends these to the Builder and waits for evidence. Then it judges. In EDD terms, the Examiner insists on executed evidence — real runs, real output, real measured values — over generative evidence, where the AI merely narrates what it believes would happen. If an expectation is unmet or the evidence is unconvincing, the Examiner returns a verdict saying what still fails, and the Builder iterates. This expectation → evidence → verdict loop repeats until every expectation is satisfied.

The striking consequence, straight from EDD: the code may carry no unit tests at all. The expectations and their evidence — recorded permanently in the audit log — are the proof of correctness. The Examiner is the test oracle, and the ledger is the test report.

5. The Builder — the only one who touches code

The Builder is the implementer, and the only agent that writes and runs real code. It receives expectations from the Examiner and produces evidence that they now hold: it builds, it runs the program, it captures screenshots and measured outputs, and it reports back which expectations are fulfilled — and only that. It is forbidden from leaking implementation detail upward; “I used a hash map keyed by cell coordinates” is a contract violation. The correct report is “E2 now holds; here is the run that shows it.”

The Builder talks to no one but the Examiner. It doesn’t know who the Owner is or what the roadmap looks like. It knows expectations, and it knows how to produce evidence. That narrowness is its strength.

6. The Sentinel — the auditor outside the chain

The five-station line is a closed pipeline, but pipelines drift. Over a long run, the Builder starts slipping code internals into its evidence; the Interpreter starts smuggling implementation hints into a behaviour. Who watches the contracts?

The Sentinel does. It stands outside the chain — not a link in it — and is the one party allowed to read the entire conversation. It periodically audits every message against its edge’s contract (does this builder→examiner message leak implementation detail? does this analyst→examiner message prescribe a solution?), writes findings to an audit log, and — by design — may message any agent directly with an advisory, a warning, or a directive to pull it back on-contract. In practice it earns its keep: it will catch the Builder naming functions in its evidence and the Interpreter drifting into solutioning — corrections a single mega-prompt would never surface.

The Sentinel never blocks or rewrites messages. It observes, reports, and nudges. It is the method’s conscience.

How they communicate: files you can replay

There is no server and no message broker. Every message is a small JSON file written to a mailbox directory, and simultaneously appended as one line to a single, gap-free, append-only ledger. That ledger is the system of record. You can replay the entire conversation message by message, filter by agent, and trace any decision’s full lineage — from the Owner’s first sentence to a specific commit — because every message carries the id of the message it replies to.

The rules live in one place: a topology file that fixes which edges exist and which message types are legal on each edge. A wrong neighbour or a wrong message type is rejected before anything is written. This is “who-talks-to-whom” enforced as data, not as etiquette.

The message vocabulary mirrors the abstraction ladder:

• Owner ↔ Interpreter: problem, clarification, roadmap, roadmap-verdict, increment, continue-query, feedback, result, question.

• Interpreter → Analyst: behaviour-to-implement.

• Analyst → Examiner: behaviour.

• Examiner → Builder: expectation, verdict.

• Builder → Examiner: evidence.

• Examiner → Analyst → Interpreter: behaviour-status (climbing back up, re-translated at each step).

• Sentinel → any agent: advisory, warning, directive.

The rhythm of one behaviour

Interpreter --behaviour-to-implement--> Analyst
Analyst     --behaviour-------------->   Examiner
Examiner    --expectation------------>   Builder    (E1..En + an integration expectation)
Builder     --evidence--------------->   Examiner
Examiner    --verdict (if unmet)----->   Builder    ⟲ loop until every expectation holds
Examiner    --behaviour-status------->   Analyst
Analyst     --behaviour-status------->   Interpreter
Interpreter --increment + continue?-->   Owner

Each agent runs in its own session and stays idle — costing nothing — until a tiny dispatcher notices a message waiting in its inbox and wakes it. Work flows by itself; the human only re-enters at the gates.

Why the constraints are the point

It’s tempting to see the neighbours-only rule and the per-edge vocabularies as friction. They are the opposite — they are what make the system legible:

• Clean abstraction boundaries by construction. Detail cannot leak up; framing cannot leak down. The structure of the conversation enforces separation of concerns better than any code review.

• A complete, tamper-evident audit trail. The ledger is append-only with a gap-free sequence, so a missing or out-of-order message is itself a signal. Commit it alongside the code and you have a permanent record of how the software was produced, not just what was produced.

• Resilience. Because all state is inspectable files, a stuck chain is just a message sitting in an inbox. A crash or restart loses only the running processes; relaunch the agents and they pick up exactly where they left off.

• Specialization over a bigger brain. The win isn’t a smarter agent — it’s a team of narrow agents communicating under strict, auditable contracts.

We put it to the test by building ThreeDeeBlocks, a 3D Tetris clone compiled to WebAssembly, end to end. The swarm worked through dozens of behaviours largely autonomously, leaving a complete trail of every expectation, every piece of executed evidence, and every course-correction — including the Sentinel flagging contract drift in real time, and the Interpreter surfacing an honest engineering substitution (a different toolchain) to the Owner for sign-off rather than burying it.

Conclusions

The Relay Method is a bet that the path to trustworthy AI-built software is not a single, ever-more-capable agent, but a structured conversation between modest ones — constrained so tightly that the structure itself does much of the quality work.

Three ideas do the heavy lifting:

1. A linear chain of single-purpose agents, each speaking only to its neighbours, so every handoff is a clean translation across abstraction levels.

2. Expectation-Driven Development at the core, so correctness is proven by plain-language expectations and executed evidence rather than asserted — sometimes with no unit tests in the code at all.

3. A file-based, append-only audit trail plus an external Sentinel, so the whole thing is inspectable, replayable, and self-policing.

The human comes out changed too: not an author hunched over a keyboard, but an editor who states problems, reviews increments, and decides. If you’ve ever wished you could see how an AI arrived at a piece of code — and trust that it actually does what it claims — the Relay Method is one concrete way to get there.

References & further reading

• Expectation-Driven Development — the practice the Examiner is built on (expectations, executed vs. generative evidence, stabilizing into tests): https://a4al6a.substack.com/p/expectation-driven-development-a

• Conway’s Law (Melvin Conway, 1968) — that systems mirror the communication structure of the organizations that build them; the Relay Method designs that communication structure deliberately.

• Behaviour-Driven Development (Dan North) — a precursor in spirit: describing behaviour in structured natural language. EDD relaxes its framework constraints for human–AI collaboration.

• The test oracle — the classic testing notion of an authority that decides whether an output is correct; in this method, the Examiner plays that role and the ledger records its verdicts.

• Continuous Delivery / Modern Software Engineering (Dave Farley) — on working in small, verifiable increments behind explicit gates, echoed here in the roadmap-and-increment rhythm.

A reader of this newsletter, who runs a team that practises trunk-based development with TDD and pair programming, recently received feedback from an ISO auditor. Nothing in the team’s system of record explicitly recorded that a second pair of eyes had verified each change, nor whose eyes those were. He saw two options: log the second pair of eyes in commit messages, or fall back to short-lived pull requests with a single approval.

Both framings miss the point. The auditor’s complaint is correct on a narrow procedural matter, and wrong on the underlying mechanism. The answer is concrete, and made of five pieces: a written policy, structured commit trailers, automated enforcement with a tight exception path, supplementary telemetry, and a sampling audit with a defined protocol. Together they form a control that is testable, deliberate, and honest about its limits.

A reader’s question

The reader wrote a careful letter. His team had moved to trunk-based development with pair programming. They were happier and produced better software. Then the auditor noticed that the team’s working practice was not visible in any system of record. He suggested co-authored commits or a return to pull requests. He wanted advice.

The answer matters beyond one team. Many engineering leaders who would otherwise drop pull requests hesitate at the audit stage, and many keep pull requests precisely because they assume audit compliance demands them. That assumption is wrong, and it is worth dismantling carefully so the alternative becomes something a serious auditor will sign off, not a hopeful sketch.

The category error

ISO does not require pull requests. Neither does SOC 2. Neither does the great majority of compliance regimes that ordinary software teams find themselves under.

ISO/IEC 27001:2022 has Annex A controls covering change management and secure development. Their text, in plain language, requires that changes are authorised, that they are tested, that environments are separated, and that the process is documented. SOC 2 has CC8.1, which requires that the organisation manages changes through a process that includes authorisation and testing. The phrase “pull request” appears in neither. The shape of the review is left to the team.

A pull request is one possible implementation of a change-review control. A pairing session followed by a co-authored commit is another. A signed-off patch series sent through email, in the style the Linux kernel has used for two decades, is a third. The standard cares about the control objective. It is the auditor’s job to assess whether the team’s chosen mechanism meets that objective, regardless of its shape.

What a control actually needs

A review control that an auditor will accept has four properties. It is defined in writing, so the auditor knows what to test against. It is implemented in practice, so what the writing says is what actually happens. It produces evidence, so the implementation can be inspected after the fact. And it is enforced, so bypassing it is hard, or at least visible.

Mature controls have all four. Weak controls have three of the four, and it is usually the implementation that is missing. This is the awkward fact at the heart of the typical pull request workflow.

The implementation gap in pull request review

A pull request creates an artefact. It does not, on its own, create a review. The artefact and the review are independent variables. A diff is opened. Someone clicks approve. The change merges. Did the approver read the diff? Did they understand it? Did they push back on anything? The artefact does not tell us. The auditor sees a green tick and is satisfied. The control objective, in substance, is unmet.

I am not arguing that all pull request review is theatre. Some teams review carefully and reject changes that need rework. The argument is narrower. The artefact does not distinguish between deep review and rubber stamping, so the artefact alone is not evidence that review happened. When the artefact is the evidence, the control is weak even where the practice is strong.

Pair programming is the opposite trade. Two engineers engage with the code in real time, arguing about names, test design, edge cases, and whether the change should exist at all. The review is continuous, and it shapes the code as it is written rather than inspecting it afterwards. The findings in Accelerate and the annual State of DevOps reports indicate that high-performing teams favour smaller batches, more frequent integration, and trunk-based development. The kind of continuous review that pairing makes possible fits the pattern those findings describe better than long-lived branches do.

The catch is that pairing leaves no native artefact. The substance is present, the artefact is missing. The auditor’s complaint is correct on this narrow point. The fix is not to throw away the substance. The fix is to add the artefact.

The composite that closes the gap

Five pieces, working together, close the implementation and evidence gap that pure pairing leaves open. Compared with a typical pull request workflow, the composite is stronger on the question that matters most for safety (whether real review happened) and weaker on the question of independent-action provability that pull requests answer through platform authentication. That trade is worth making for most teams under ISO 27001 and SOC 2, but the article will be honest about both sides.

The policy

The team’s change management policy must say, in plain language, that pair programming or mob programming is the team’s primary control for the four-eyes requirement on code changes, that every change reaching the trunk is produced by at least two engineers working together, and that the second engineer is recorded on the commit. Without this document, an auditor has no benchmark against which to test the team’s behaviour. With it, every other artefact has meaning. Half a page, signed by the engineering lead and reviewed annually, is sufficient.

The commit trailers

Git supports structured trailers natively, and the conventions are not new. The Linux kernel has used Signed-off-by, Reviewed-by, Tested-by, and Acked-by for two decades to attribute who did what to a patch. The same machinery is available to any team. A pairing team adopts Co-authored-by for the active partner and, where appropriate, Reviewed-by for an additional asynchronous reviewer. A typical commit looks like this.

feat(billing): apply VAT for cross-border invoices

Co-authored-by: Jane Doe <jane@example.com>
Reviewed-by: Marco Rossi <marco@example.com> (commit b3f7a2c)

These trailers are machine-readable. A script can count them, sample them, and verify their format. They are also human-readable: an auditor opening the commit sees, at a glance, who participated.

The enforcement

A pre-commit hook on the developer’s machine refuses commits that do not meet the rules. A CI check on the integration branch is the backstop, in case the local hook is bypassed. The rules need to be specific, otherwise enforcement is theatre too.

A defensible minimum has five rules. Every commit on the integration branch must contain at least one Co-authored-by or Reviewed-by trailer. The trailer’s email must resolve against the company directory, so anonymous attestation is impossible. The author and the named co-author or reviewer must be different identities. A commit without a co-author must contain an explicit Solo-work: line naming an asynchronous reviewer by directory identity and the commit hash that reviewer inspected. A commit lacking any of the above is rejected by CI, and the rejection is logged.

The exception path is deliberately narrow. There is no zero-review path. Either the work was paired (Co-authored-by), or it was reviewed asynchronously (Reviewed-by), or it was solo with an explicit asynchronous reviewer attached to a specific commit hash (Solo-work:). All three produce structured evidence; none allows a change to reach the trunk unattested.

The telemetry

Where pairing tools such as Tuple, Pop, or mob.sh are in use, their session records corroborate the trailers. Calendar invites for recurring mob sessions add a second source. None of this is the primary evidence; it is corroboration, and it makes systematic falsification harder. The composite does not lean on telemetry, because individual sources are easy to dismiss in isolation. The trailers and the audit do the work.

The sampling audit

This is the linchpin of the composite, and it deserves a defined protocol rather than a vague intent.

A defensible minimum specification: each quarter, draw a random sample of commits from the integration branch using a seeded selection script. The sample size is the greater of ten percent of commits in the period or twenty commits, capped at fifty. The audit is run by a person who is not the engineering lead of the audited team, such as a peer team lead, a quality engineer, or an internal audit function, in order to avoid self-review. For each sampled commit, both named engineers are interviewed separately, with three standard questions: what was the purpose of this change, what alternatives were considered, and what would you do differently now. The interviewer records answers in a shared log alongside the commit hash and the date.

Failure criteria are explicit. If two or more sampled engineers cannot recall a change they are recorded on, the result is a process finding that triggers a documented remediation: a refresh of the policy, a refresh of the team’s pairing protocol, or, in serious cases, a return to short-lived pull requests until the team rebuilds the practice. The criterion does not catch deliberate fraud reliably; it does catch process drift, which is the failure mode most relevant under ISO 27001 and SOC 2.

This is the step that converts a culture of trust into a control an external auditor can test, because it gives the auditor a mature internal control to inspect rather than a claim to take on faith. Auditors recognise this structure. It is the same shape they apply to financial controls.

Three honest objections

A serious reader will already have noticed three problems. Let me address them rather than pretend they do not exist.

Self-attestation

A Co-authored-by trailer is, in the end, a string the committer types. A pull request approval is stronger on this narrow point: it is tied to an authenticated user action with its own audit trail. The composite does not pretend to match that property.

What the composite does instead is shift the weight of the control from a preventative mechanism (a click that allegedly happens before merge) to a detective mechanism (a sampling audit that verifies, after the fact, that review happened in substance). This is a recognised pattern in audit design: detective controls are weaker on the moment of action and stronger on the cumulative behaviour of the system. Two engineers conspiring to falsify a trailer must also conspire to give consistent, detailed answers to separate interviews about a commit they did not work on, and they must do so every time a sampling audit lands on a falsified commit. Casual misuse becomes uneconomic; deliberate fraud becomes detectable through pattern analysis.

The composite does not eliminate the self-attestation problem. It raises the cost of circumvention to the point where the dominant residual risk is process drift, not collusion. For ISO 27001 and SOC 2, that is sufficient. For regimes that demand non-repudiation as a primary property of the control, it is not, and the regulatory carveouts below apply.

Distributed teams

Pairing across timezones is genuinely hard. A team that cannot pair synchronously cannot rely on pairing as its primary control without modification. The honest answer is a hybrid: synchronous pairing within a timezone, asynchronous review with Reviewed-by trailers across timezones, and a policy that distinguishes the two cases. The asynchronous case shares structure with a pull request workflow, but the review is recorded in the commit history rather than in a separate workflow, the integration is not delayed by it, and the same enforcement and sampling apply. The article does not claim distributed teams can drop the second pair of eyes. It claims they can record it without inheriting the bottleneck of long-lived branches.

Solo work

Some changes are genuinely solo: a quick fix, a documentation typo, a configuration tweak that does not warrant a pairing session. The exception path in the hook handles this, but only with a name and a commit hash attached, and only with the same sampling treatment as paired commits. The team’s policy should set a threshold above which pairing is required regardless. This is no different from the way mature pull request workflows distinguish trivial changes from substantive ones; the difference is that the distinction is recorded in the commit, not in the branch lifecycle.

The regulatory carveouts

There are regimes where the literal separation of author and approver is required, and where genuine pairing cannot satisfy the requirement on its own. SOX controls over financially material systems, FDA rules for software in medical devices, parts of the GxP family in pharmaceutical manufacturing, and certain defence-sector standards come to mind. In those contexts, the team can still pair as its way of working, but a separate, documented approval step by a person who was not part of the pairing is also required. The composite described above is not a universal substitute. It is a substitute for the great majority of teams under ISO 27001 and SOC 2, which is the relevant audience here. A team operating under one of the stricter regimes should read the actual control text and design accordingly, and should be wary of any blog post, including this one, that promises a single mechanism for all cases.

Talking to the auditor

The conversation with the auditor is a conversation about controls, not artefacts. Open by asking which control objective the auditor is testing, in their own words. Almost always, the answer reduces to “evidence that a person other than the author has reviewed the change before it reached production”. Once that objective is on the table, the team presents the policy, the trailers, the enforcement rules, the telemetry, and the sampling audit log as the answer. Bring the policy. Bring an example of a sampled commit with the corresponding interview record. Treat the auditor as a serious reader who needs to be shown a serious control.

Most auditors respond well to a team that has thought clearly about what the standard is asking for and has produced a deliberate answer. Some do not. If the auditor insists on pull requests as a fixed shape, regardless of the control objective they implement, the team has three options. First, escalate within the audit firm: some firms have specialists who understand modern engineering practice, and some auditors can be replaced. Second, accept the audit firm’s preference and run a pull request workflow alongside the team’s real practice, knowing that the pull request is a record-keeping ritual rather than the actual review. Third, move the certification to a different firm.

None of these is comfortable, and the article will not pretend they are. Escalation is not always available, especially for smaller customers of larger audit firms; switching firms is disruptive and rarely fast; running a parallel ritual erodes the cultural commitment to pairing over time. The honest position is that a team should attempt the first option first, prepare for the second as a fallback during the certification cycle in question, and consider the third only if the audit relationship cannot be repaired.

The choice

The reader’s question looked technical. It is, in the end, a choice between two kinds of compliance.

One kind is built from the appearance of review. It is cheap, recognisable to a tired auditor, and satisfies the letter of the standard at the cost of producing very little of the substance the standard intends. The other kind is built from real review, made auditable. It costs discipline, a small amount of tooling, and a defined sampling protocol. It produces both real safety and a paper trail, and it makes the team better at building software along the way.

I know which one I think is worth buying. So does the reader, which is why he wrote in the first place. The good news is that he does not have to abandon what is working in order to satisfy an auditor. He has to write the policy, install the hooks, define the enforcement rules, set up the sampling protocol, run the audits, and have the conversation with care. The pairing stays where it belongs, in front of the code, with two pairs of eyes that actually see it.

References

1. A. Laforgia, Stop Using Pull Requests, https://a4al6a.substack.com/p/stop-using-pull-requests

2. A. Laforgia, The Illusion of Compliance, https://a4al6a.substack.com/p/the-illusion-of-compliance

3. ISO/IEC 27001:2022, Information security, cybersecurity and privacy protection. Information security management systems. Requirements, Annex A controls on change management and secure development.

4. AICPA, Trust Services Criteria for Security, Availability, Processing Integrity, Confidentiality, and Privacy, criterion CC8.1 on change management.

5. Linux kernel project, Submitting patches: the essential guide to getting your code into the kernel, Documentation/process/submitting-patches.rst, on the use of Signed-off-by, Reviewed-by, Tested-by, and Acked-by trailers.

6. Git documentation on commit trailers and the Co-authored-by convention as supported by GitHub and GitLab.

7. P. Hammant and contributors, Trunk Based Development, https://trunkbaseddevelopment.com

8. N. Forsgren, J. Humble, G. Kim, Accelerate: The Science of Lean Software and DevOps, IT Revolution Press, 2018.

9. K. Beck, Test-Driven Development: By Example, Addison-Wesley, 2002.

10. M. Feathers, Working Effectively with Legacy Code, Prentice Hall, 2004.

The “AI killed Agile” argument confuses two different problems. There is a speed problem — how fast can we turn an idea into working software? And there is a knowledge problem — how do we figure out which idea is the right one? AI largely solves the first. It does not touch the second. Agile was created for the second. The push for “write a detailed spec, then have AI build it” is not innovation — it is waterfall with faster computers. And the person who (accidentally) invented waterfall warned us it would fail.

The principle that built many Agile practices — learn what to build by building, observing, and adapting — is more relevant than ever. When building is nearly free, the only reason not to iterate is if you never understood why iteration mattered in the first place.

1. The Argument (and Why It Feels Right)

A new narrative is gaining traction: AI has made Agile obsolete.

Steve Jones, Executive VP at Capgemini, puts it bluntly: “Agentic SDLCs are too fast for Agile.” Jennifer Jones-Mitchell of HumanDrivenAI argues that “Agile’s iterative cycles still rely on humans to ideate, test, and refine over weeks or months. Generative AI tools can produce viable prototypes, campaigns, or solutions in minutes” [6, 15].

The argument is seductive, and it would be dishonest not to acknowledge why. Anyone who has sat through a performative sprint planning meeting, a daily standup that was just a status report, or a retrospective that changed nothing feels the appeal. If AI can ship in a day what used to take a sprint, why keep the ceremony?

That frustration is legitimate. Martin Fowler, one of the Agile Manifesto’s signatories, coined “semantic diffusion” to describe how Agile lost its original meaning. Many organisations practice what he calls “faux-agile”: implementing Scrum ceremonies while ignoring the adaptive principles that made those ceremonies useful [4].

2. The Confusion: Speed vs. Knowledge

The “AI killed Agile” argument treats the speed of coding as the constraint Agile was designed to address. It wasn’t.

In February 2001, seventeen practitioners met at Snowbird, Utah — representing Extreme Programming, Scrum, Crystal, Feature-Driven Development — united against heavyweight, documentation-driven processes that were failing to deliver working software [3]. The result was the Agile Manifesto:

• Responding to change over following a plan

• Working software over comprehensive documentation

• Customer collaboration over contract negotiation

• Individuals and interactions over processes and tools [1]

None of these say “write code faster.” They address a different problem: we do not know what to build until we start building it. Not because we are bad at planning, but because the information needed to plan correctly does not exist until real users interact with real software.

This is an epistemological claim, not a methodological preference. The Manifesto’s Principle 2 states: “Welcome changing requirements, even late in development.” Principle 11: “The best architectures, requirements, and designs emerge from self-organising teams” [2]. The word emerge is doing heavy lifting. Requirements are not specified and then executed. They are discovered through building, delivering, and observing.

Research bears this out: 40-60% of software project failures originate in requirements, not slow implementation [17]. Only about 20% of features deliver the positive impact initially intended [14]. The problem was never “we code too slowly.” It was always “we don’t know which features are the right ones until we put them in front of real users.”

AI makes coding faster. It does not make stakeholders know what they want. It does not reduce market uncertainty. It does not reveal whether users will actually use what you built.

3. The Inversion: Faster Means More, Not Fewer

Here is the part the “AI kills Agile” crowd gets exactly backwards.

If AI makes it cheaper to produce working software, the logical conclusion is not “we don’t need iterations.” It is “we can afford more iterations.”

Eric Ries’s Lean Startup methodology is built on the Build-Measure-Learn loop: build the smallest thing that tests your assumption, measure what happens when real users interact with it, learn whether to continue or pivot. The antidote to uncertainty is not better planning — it is “validated learning: a rigorous method for demonstrating progress when one is embedded in the soil of extreme uncertainty” [8].

If AI collapses the “Build” phase from weeks to hours, a team can run more Build-Measure-Learn cycles per month. That is not the death of Agile. It is Agile unleashed.

But here is the nuance: AI makes Build so cheap it rounds to zero. It does not make Measure or Learn cheap. Observing how real users behave, understanding why they behave that way, synthesising that understanding into the next decision — these remain expensive, slow, and irreducibly human. So the bottleneck shifts. In a world of instant builds, the constraint moves from “how fast can we code?” to “how fast can we learn?” That does not kill iteration. It recenters it on what always mattered: the learning.

Kent Beck, co-author of the Agile Manifesto, sees it this way. AI has shifted the cost landscape: “The whole landscape of what’s ‘cheap’ and what’s ‘expensive’ has all just shifted.” His recommendation: test approaches previously deemed too expensive. Not “abandon discipline.” Not “skip feedback.” Experiment more [5].

4. “But I Can Iterate on the Spec”

There is a smarter version of the “AI kills Agile” argument that deserves engagement. It goes: “I write a spec, AI builds a prototype in minutes, I show it to users, I revise the spec, AI rebuilds. The loop is tight and fast. That IS iteration.”

This is reasonable, and the workflow it describes can genuinely work — when the spec is treated as a disposable hypothesis. Write a one-page prompt, generate a prototype, show it to users, throw the prompt away, start fresh. That is rapid experimentation with a generative tool. Call it agile if you want. It might well be.

But that is not what most people mean by “spec-driven development,” and it is not what the “write a solid spec upfront” advocates are proposing. Their version is different: invest in a comprehensive, detailed specification — a document that captures requirements, edge cases, business rules — then hand it to AI for execution. The spec is the artifact that is reviewed, signed off, refined, and version-controlled. It becomes the primary record of truth.

And the moment that happens, the spec takes on a gravity of its own. Each revision carries forward assumptions from previous iterations that nobody re-examines. The spec becomes a sedimentary record of decisions, many of which were wrong but are now buried under layers of refinement. You are no longer exploring. You are predicting — encoding guesses about what users need into a document and then executing those guesses with increasing fidelity.

That is waterfall. It has always been waterfall. It is what killed the FBI’s $170 million Virtual Case File — not slow coding, but a specification that could not keep up with what the organisation was learning about its own needs [9].

Winston Royce — the person credited with inventing waterfall — understood this in 1970. In the same paper, he warned: “I believe in this concept, but the implementation described above is risky and invites failure.” He advocated passing through the process “at least twice” [16]. The waterfall model as practiced was a misreading of his paper. He presented it as a cautionary example.

Fifty-five years later, “write a detailed spec and have AI build it” recreates the same pattern. The spec is still wrong. It will always be wrong. Because requirements are not merely vague — they are unknown and emergent. The Manifesto says it plainly: “Welcome changing requirements, even late in development” [2]. Not “write better requirements.” Welcome the change, because the change is the learning.

The distinction is precise: specs are predictive — you guess what users need and encode the guess. Working software in users’ hands is explorative — you discover what users actually do. A spec, no matter how rapidly iterated, is a model of what users want. Working software is what users interact with. The gap between model and reality is where projects fail.

5. The Real Threat

The “AI killed Agile” narrative has a political dimension that makes it genuinely dangerous.

One reason Agile became bureaucratic was that management wanted predictability. Sprints, story points, velocity charts gave managers the illusion of control over an inherently uncertain process. Agile was adopted by organisations that wanted its speed without accepting its core bargain: you cannot predict what you will deliver until you start delivering it.

AI makes this worse. The pitch is irresistible to a certain kind of stakeholder: “Just write the spec. The AI will handle it.” This is not a technology argument. It is a power argument — the desire to return to a world where management defines requirements, engineering executes them, and the humbling process of learning from users can be skipped.

Healthcare.gov is the canonical example. Over $2.1 billion spent. Six users on launch day. The team spent months building against frozen requirements that were wrong. AI would have generated the wrong code faster — and the team still would have discovered the integration failures at the same time, because they deferred testing until the end. The post-mortem recommended “incremental approaches such as betas, early testing and regular delivery” [10, 11]. The problem was not development speed. It was the organisational refusal to learn incrementally. Faster tools do not fix a process designed to avoid learning.

The real threat is not that engineers will stop iterating. Most good engineers know better. The threat is that AI gives non-technical decision-makers a fresh justification for the approach they always preferred: define everything upfront, hand it off, and skip the uncomfortable discovery that the original vision was wrong.

Alistair Cockburn, another Manifesto co-author, distilled Agile to four imperatives: Collaborate, Deliver, Reflect, Improve [7]. Each one is a check on the natural organisational tendency to plan in isolation, build in darkness, and declare victory. AI does not make any of these obsolete. It makes the temptation to skip them more seductive and the consequences of skipping them more expensive.

Conclusions

No technology has ever solved the problem of building something nobody wanted. Faster horses, faster compilers, faster AI — the tool changes, the failure mode does not.

AI makes building cheaper. That is genuinely transformative. But when building is nearly free, learning what to build becomes the only thing that separates teams that ship products people use from teams that ship products nobody asked for.

Many Agile practices should evolve (“We are uncovering better ways of developing software by doing it and helping others do it”) but the core discipline survives: build something small, put it in front of real people, learn from what happens, adapt. The loop does not get shorter with AI. It gets cheaper. And when the loop is cheaper, you should run it more often, not less.

AI did not kill Agile. AI killed the excuse for not iterating.

References

[1] Beck, K. et al. “Manifesto for Agile Software Development.” AgileManifesto.org, 2001.

https://agilemanifesto.org/

[2] Beck, K. et al. “Principles behind the Agile Manifesto.” AgileManifesto.org, 2001. https://agilemanifesto.org/principles.html

[3] Agile Manifesto Authors. “History: The Agile Manifesto.” AgileManifesto.org, 2001. https://agilemanifesto.org/history.html

[4] Fowler, M. “Agile Software Guide.” MartinFowler.com. https://martinfowler.com/agile.html

[5] Orosz, G. “TDD, AI agents and coding with Kent Beck.” The Pragmatic Engineer.

The Pragmatic Engineer

TDD, AI agents and coding with Kent Beck

Stream the Latest Episode…

Listen now

a year ago · 215 likes · 1 comment · Gergely Orosz

[6] InfoQ. “Does AI Make the Agile Manifesto Obsolete?” InfoQ, February 2026. https://www.infoq.com/news/2026/02/ai-agile-manifesto-debate/

[7] Cockburn, A. “Heart of Agile.” HeartOfAgile.com.

https://heartofagile.com/

[8] Ries, E. “The Lean Startup - Principles.” TheLeanStartup.com. https://theleanstartup.com/principles

[9] Goldstein, H. “Who Killed the Virtual Case File?” IEEE Spectrum, September 2005. https://spectrum.ieee.org/who-killed-the-virtual-case-file

[10] CIO. “6 Software Development Lessons From Healthcare.gov’s Failed Launch.” CIO.com, 2013. https://www.cio.com/article/288541/developer-6-software-development-lessons-from-healthcare-gov-s-failed-launch.html

[11] NPR. “This Slide Shows Why HealthCare.gov Wouldn’t Work At Launch.” NPR, November 2013. https://www.npr.org/sections/alltechconsidered/2013/11/19/246132770/this-slide-shows-why-healthcare-gov-wouldnt-work-at-launch

[12] Springer. “Continuous clarification and emergent requirements flows in open-commercial software ecosystems.” Requirements Engineering, 2016. https://link.springer.com/article/10.1007/s00766-016-0259-1

[13] ScienceDirect. “Tackling Requirements Uncertainty in Software Projects: A Cognitive Approach.” 2021. https://www.sciencedirect.com/science/article/pii/S2666307421000218

[14] DevIQ. “Big Design Up Front (BDUF): A Software Development Antipattern.” DevIQ.com. https://deviq.com/antipatterns/big-design-up-front/

[15] Jones-Mitchell, J. “How AI Killed the Agile Process (And Why That’s a Good Thing).” HumanDrivenAI, December 2024. https://humandrivenai.com/2024/12/04/how-ai-killed-the-agile-process-and-why-thats-a-good-thing/

[16] Royce, W.W. “Managing the Development of Large Software Systems.” Proceedings of IEEE WESCON, 1970.

[17] Requiment. “Why Do Software Development Projects Fail?” Requiment.com. https://www.requiment.com/why-do-software-development-projects-fail/

And AI-assisted development is about to make that imbalance much, much worse.

I’ve been thinking about this as a simple two-by-two grid. On one axis: does the system do the thing, or not? On the other: should it?

That gives us four quadrants:

Functional: the system should do it, and it does. This is the happy place. Our login form authenticates users. Our checkout calculates the right total. We wrote a test, the test passes, everyone goes home.

Missing: the system should do it, but it doesn’t. This is where backlogs live. Features not yet built. Edge cases not yet handled. It’s a known gap, and known gaps are manageable. We can prioritise them, plan for them, write a user story.

Faulty: the system should not do it, but it does anyway. This is where things get uncomfortable.

Safe: the system should not do it, and it doesn’t. This is where things get invisible.

Most of our engineering effort goes into the top half of this grid. We write requirements that describe what the system should do. We write tests that verify it does those things. We track missing functionality in backlogs and roadmaps. We have entire events dedicated to deciding what to build next.

But the bottom half? That’s where the real trouble lives. And it’s the half that AI coding tools don’t touch.

AI is brilliant at the top half

Let’s be fair. AI-assisted development is genuinely transforming the functional and missing quadrants.

Need to build a login form? An AI coding agent can produce one in seconds, complete with validation, error handling, and tests. Got a backlog of missing features? AI can chew through them faster than any team could before. The top half of the grid is where AI shines, because it’s a space defined by clear, expressible intent. We can describe what we want. The model can generate it. We can verify it works.

This is real progress. Teams are shipping faster. Backlogs are shrinking. Features that would have taken days are landing in hours. If our definition of productivity is “how quickly can we build the things we’ve decided to build,” then AI is a genuine leap forward.

But productivity isn’t the same as quality. And quality lives in all four quadrants, not just the top two.

Why faulty is so hard to find

The fundamental problem with faulty behaviour is that we can’t write a list of everything a system shouldn’t do. The space is, for all practical purposes, infinite.

We can write a requirement that says “the system should let users reset their password.” We can test that. But can we write a requirement that says “the system should not let users reset someone else’s password”? We could, but that’s just one of a thousand things the password reset flow shouldn’t do. It also shouldn’t expose the reset token in the URL. It shouldn’t send the new password in plain text. It shouldn’t allow a reset request to be replayed. It shouldn’t let someone enumerate valid email addresses by observing response times.

Each of those is a specific “should not” that someone had to think of. And someone had to think of it because, at some point, a system somewhere did exactly that.

This is why security vulnerabilities are so hard to eliminate. A vulnerability is almost always a case of the system doing something it was never intended to do. Not a missing feature. Not a broken feature. An extra, unwanted behaviour that emerged from the interaction between components that individually work fine.

And that’s the second reason faulty behaviour is hard to catch: it’s emergent. Our authentication module works. Our session management works. Our API gateway works. But the specific way they interact under a particular sequence of requests creates a behaviour that none of them were designed to produce. We can’t find it by testing each component in isolation. We can only find it by understanding the system as a whole, and the system as a whole is more complex than any single person can hold in their head.

There’s also a subtler problem. Faulty behaviour often looks like correct behaviour from the inside. The system is doing exactly what the code tells it to do. It’s following its instructions perfectly. The fault isn’t in the execution; it’s in the gap between what we told the system to do and what we meant for it to do. Every line of code is an instruction the system will follow faithfully. We just didn’t always think through the consequences of every instruction we gave.

Why AI makes the bottom half worse

Here’s the thing that nobody in the “AI will 10x your productivity” conversation wants to talk about: every line of code an AI generates is also a line of code that might do something we didn’t ask for.

When a human writes code slowly, there’s a natural friction that acts as a filter. We think about what we’re writing. We consider edge cases. We’ve been burned before by a particular pattern, so we avoid it. We have institutional memory, battle scars, and instincts shaped by years of production incidents.

An AI coding agent has none of that. It generates code that satisfies the stated requirement, and it does so fluently and confidently. But it has no concept of “should not.” It wasn’t there when our system went down at 3am because of a race condition. It doesn’t know that our payment provider’s API behaves differently in sandbox mode. It doesn’t understand that the specific way it structured a database query makes it vulnerable to timing attacks.

An AI can produce a perfectly functional password reset flow that also happens to leak information through response times. Every test passes. The feature works. The faulty behaviour is invisible to the test suite because nobody thought to test for it, and the AI certainly didn’t flag it.

And the volume problem makes this worse. If AI lets us ship ten times more code, we’ve also shipped ten times more surface area for faulty behaviour. The top half of the grid fills up faster. The bottom half gets more dangerous at exactly the same rate.

Why safe is so hard to confirm

If faulty behaviour is hard to find, safe behaviour is hard to even think about. How do we verify the absence of something? How do we test that something doesn’t happen?

We can test that a non-admin user cannot access the admin panel. That’s a specific assertion about a specific “should not.” But the number of things a non-admin user should not be able to do is essentially unbounded. They shouldn’t be able to modify other users’ data. They shouldn’t be able to trigger a database export. They shouldn’t be able to craft a request that causes the server to leak its environment variables. Each of these is a test we could write, but only if we’ve already imagined the specific threat.

This is the core difficulty: the safe quadrant is defined by absence, and absence is invisible. Nobody notices the things a system doesn’t do until it starts doing them. The safe quadrant only becomes visible when something moves out of it and into the faulty quadrant. By then, we’ve got a production incident.

There’s also the problem of regression. Something that was safely excluded yesterday can quietly creep in today. A new dependency introduces a behaviour we didn’t ask for. A refactor changes the boundary conditions. A configuration change opens a path that was previously closed. The safe quadrant isn’t stable. It needs constant maintenance, but because we can’t see it, we don’t know when it’s eroding until something breaks through.

AI-generated code accelerates this erosion. When an AI agent refactors a module or introduces a new dependency, it’s optimising for the stated goal. It’s not thinking about what the previous version of the code was quietly preventing. The old code might have been clunky, but its clumsiness might have been the very thing keeping a dangerous path closed. The AI tidies it up, the tests still pass, and something that was safe is now faulty. Nobody notices until it matters.

What this means for how we work

Most of our testing practices are built for the top half of the grid. Unit tests verify that functions produce the right output for given inputs. Acceptance tests verify that the system exhibits the behaviours described in user stories. Both are firmly in the “should do” world.

To address the bottom half, we need different tools and, more importantly, a different mindset.

Property-based testing is one approach. Instead of testing specific inputs and outputs, we define invariants that should hold across all possible inputs. “No matter what string we pass to this function, the output should never contain unescaped HTML.” That’s a “should not” expressed as a universal constraint. It doesn’t cover everything, but it covers more than a handful of example-based tests.

Observability helps too. If we can’t enumerate everything the system shouldn’t do in advance, we can at least watch what it actually does in production and look for surprises. Anomaly detection, audit logging, runtime assertions. These are tools for spotting faulty behaviour after the fact, which isn’t as good as preventing it, but it’s far better than not spotting it at all.

Threat modelling is explicitly about the bottom half. We sit down and ask: what could go wrong? What could this system do that we don’t want it to? It’s a structured way of populating the “should not” column, and it’s one of the few practices that directly targets the faulty quadrant.

But perhaps the most important thing is simply to recognise that the bottom half exists. Most teams spend all their energy making the system do the right things and almost none making sure it doesn’t do the wrong things. The backlog is full of features to build. It’s never full of behaviours to prevent.

And that asymmetry is where the risk lives.

The uncomfortable truth about AI and quality

We’re quite good at building systems that do what they should. We’re not nearly as good at building systems that don’t do what they shouldn’t. And the reason is structural, not just a matter of effort. The “should do” space is finite and enumerable. The “should not” space is infinite and mostly invisible.

AI is an amplifier. It amplifies whatever we’re already doing. If we’re focused on building features, AI will help us build features faster than ever. If we’re focused on preventing unwanted behaviour, AI can help with that too. But almost nobody is focused on preventing unwanted behaviour, because our entire industry is structured around the top half of the grid.

So what AI actually amplifies, in practice, is the imbalance. More code, faster. More features, sooner. More surface area for things to go wrong, with no corresponding increase in our ability to spot them.

This doesn’t mean we should stop using AI. It means we should stop measuring AI’s impact purely by how fast it fills the top half of the grid. The real question isn’t “how quickly can we build?” It’s “how confident are we that what we built only does what it should?”

A green test suite means the system does the things we thought to check. It says nothing about the things we didn’t think to check. And when AI is writing the code, the gap between “what we checked” and “what the code actually does” grows wider with every prompt.

The four quadrants are a simple model, but they make one thing very clear: half of system quality is about what the system does, and the other half is about what it doesn’t. AI is making the first half easier and the second half harder. And until we take the second half as seriously as the first, faster will just mean faster into trouble.

Conclusions

The next time someone tells us AI will revolutionise software development, we should ask: which half? Because the half it’s good at was never the half we were struggling with.

We don’t need to build faster. We need to think wider. The quadrant that will hurt us most is the one we never thought to look at, and AI won’t look at it for us.

The most dangerous line of code isn’t the one that fails a test. It’s the one that passes every test and still does something nobody asked for. AI writes a lot of those lines now. We should probably start paying attention to that.

”How?” is a good question.

Change Advisory Boards (CABs) were designed in the late 1980s to bring order to chaotic IT environments. Nearly four decades later, the largest empirical study of software delivery performance ever conducted — spanning over 23,000 responses from more than 2,000 organizations — has delivered a damning verdict: external change approval is worse than having no change approval process at all. CABs do not reduce failures. They slow delivery, encourage riskier deployments, erode team ownership, and create a false sense of security. Yet they persist — not because they work, but because of identity threat, loss aversion, and institutional self-preservation. This article examines the evidence, explores the psychology, and argues that the deployment pipeline — powered by TDD, automated testing, and progressive delivery — is the only “change advisory board” that actually works.

TL;DR

The DORA research program (23,000+ responses, 2,000+ organizations, 2014-2024) found that external change approval by a CAB is negatively correlated with lead time, deployment frequency, and restore time, and has no correlation with change fail rate — the very metric it exists to improve. Organizations with heavyweight approval processes are 2.6x more likely to be low performers. The UK Financial Conduct Authority found that CABs in financial services approved over 90% of changes while change-related incidents remained a top cause of disruption. Even ITIL 4 — the framework that invented CABs — has made them optional. The alternative is not anarchy: it’s peer review, TDD, automated pipelines, feature flags, and team ownership. CABs survive not because they work, but because removing them triggers identity threat in the people whose roles depend on them.

1. A Solution from Another Era

In the late 1980s, the British Government had a problem. IT services were unreliable, changes were chaotic, and a single bad deployment could bring down an entire mainframe environment. The Central Computer and Telecommunications Agency (CCTA) responded by creating the IT Infrastructure Library (ITIL) — a framework that included, among many processes, the Change Advisory Board.

The CAB made sense in that world. Changes were infrequent and large — quarterly or annual releases. Infrastructure was physical and changes were difficult to reverse. There was no automated testing, no CI/CD, no easy rollback. The cost of failure was extremely high and recovery was slow. A committee of cross-functional stakeholders evaluating changes before they went live was a reasonable response to these constraints.

But that world no longer exists.

Today, elite-performing engineering organizations deploy code multiple times per day. They use automated testing suites that run thousands of checks in minutes. They deploy behind feature flags, roll out changes to 1% of users first, and roll back automatically if error rates spike. They have observability platforms that make production behavior visible in real time.

And yet, in thousands of organizations, a committee still meets once a week to review change request forms and rubber-stamp deployments.

2. What the Data Actually Says

The DevOps Research and Assessment (DORA) team — founded by Nicole Forsgren, Jez Humble, and Gene Kim — has conducted the largest and longest-running research program on software delivery performance. Their research spans multiple years (2014-2024), has collected over 23,000 survey responses from more than 2,000 organizations across all sizes and industries, and uses peer-reviewed statistical analysis.

Their findings on change approval are unambiguous:

“We found that external approvals were negatively correlated with lead time, deployment frequency, and restore time, and had no correlation with change fail rate. In short, approval by an external body (such as a manager or CAB) simply doesn’t work to increase the stability of production systems, measured by the time to restore service and change fail rate. However, it certainly slows things down. It is, in fact, worse than having no change approval process at all.” — Forsgren, Humble, Kim, Accelerate (2018)

Read that again. External approval by a CAB is worse than having no change approval process at all. This is not an opinion from a DevOps evangelist. It is the conclusion of the largest empirical study of software delivery ever conducted.

The 2019 State of DevOps Report added further specificity:

• Peer review during development, supplemented by automation, is the most effective change approval approach.

• No evidence was found supporting the hypothesis that formal, external review processes reduce change failure rates.

• Organizations using heavyweight approval processes are 2.6 times more likely to be low performers.

• When team members have a clear understanding of the change approval process, this drives higher performance — the process clarity matters more than the process weight.

3. The Regulator’s Own Data

If the DORA findings seem too theoretical, consider what happens when a financial regulator looks at the actual numbers.

The UK Financial Conduct Authority published “Implementing Technology Change” in February 2021, analyzing over 1 million production changes across UK financial institutions. What they found should alarm anyone who relies on a CAB for safety:

• CABs approved over 90% of the major changes they reviewed.

• Some firms had not rejected a single change during all of 2019.

• Despite this, change-related incidents were consistently one of the top causes of failure and operational disruption.

• Of high-severity customer-facing incidents, 24% had change-related root causes.

• The sampled firms deployed nearly 68,000 major changes over 2019, resulting in 2,600 incidents. Major changes had a failure rate of 3.8% — more than double the standard rate.

• Firms using smaller, more frequent releases had higher change success rates.

• Firms demonstrating greater adoption of agile methodologies were less likely to suffer change-related incidents.

This is a regulator’s own data confirming that CABs are rubber stamps that do not prevent the failures they exist to prevent. A 90%+ approval rate is not a quality gate — it is a formality. The FCA found the same thing DORA found: it is not committees that reduce change risk, but better engineering practices — smaller batches, more automation, more testing, more frequent deployment.

4. Eight Ways CABs Hurt You

They create bottlenecks

CABs typically meet on a fixed schedule — often weekly. A change that takes one hour to develop may wait days for approval. Stuart Rance, an ITIL practitioner, quantifies it directly: “If you hold a CAB meeting once a week, and change requests have to be submitted three days before the CAB meeting, then you delay every change by up to 10 days.”

If a CAB meets weekly and reviews 20-30 changes in an hour, each change receives approximately 2-3 minutes of scrutiny. This is neither thorough review nor fast flow — it is the worst of both worlds.

They reduce deployment frequency

Elite-performing teams deploy on demand, multiple times per day. A weekly CAB meeting caps deployment frequency at once per week at most — placing a hard ceiling on team performance.

They do NOT improve stability

This is the most damning finding. The entire justification for CABs is that they reduce the risk of production failures. The data says they don’t. External approvals had no correlation with change fail rate (DORA, 2014-2019). None.

They shift accountability away from teams

When a CAB approves a change, accountability becomes diffused. The team is no longer solely responsible — the CAB “approved” it. The feedback loop between writing code and owning its impact in production is broken. The incentive to invest in quality shifts from “I own this” to “the CAB will catch problems.”

They encourage larger, riskier deployments

This is the critical paradox. CABs are designed to reduce risk, but by reducing deployment frequency, they force teams to batch changes. Larger batches are inherently more complex, harder to test, harder to understand, and harder to roll back. CABs often increase the risk of painful deployment failures as a direct result of the constraints they impose.

“Ask a programmer to review ten lines of code, he’ll find ten issues. Ask him to do five hundred lines, and he’ll say it looks good.” — Giray Ozil

This is precisely the dynamic CABs create: large batches reviewed superficially versus small batches reviewed thoroughly.

They create learned helplessness

Engineers who must get deployments approved by committees start self-censoring innovative ideas. They tell other engineers that their features “won’t pass the committees’ checks,” creating a culture of pre-emptive surrender. This creates a vicious cycle: the CAB reduces team autonomy, which reduces team ownership, which reduces quality, which “justifies” more CAB oversight.

They inflate lead time

Elite performers have lead times of less than one hour; low performers have lead times of one to six months. Lead time is negatively correlated with external approval processes. CABs push organizations toward the latter.

They damage developer morale

Waiting for CAB approval is demoralizing for engineers who have already completed their work. The process signals distrust: “We don’t trust you to deploy your own code.” Filling out change request forms is widely perceived as bureaucratic busywork. The DORA research found that heavyweight change management processes contribute to burnout.

Nicole Forsgren noted: “The report has found year after year that a slower change management process can paradoxically result in more instability, not less.”

5. What the Industry’s Best Minds Say

The critique of CABs is not a fringe position. It comes from the most respected voices in software engineering:

Jez Humble (DORA co-founder, co-author of Continuous Delivery):

“Most change management is Change Management Theatre. It is not about making things better, it is about covering your ass when things go wrong.”

External change approval is “risk management theater” — like TSA’s enhanced airport security, “it accomplishes nothing at enormous cost, giving the impression that you’re managing the risk of making changes to the production environment, while actually making the situation worse.”

“How are outside reviewers expected to review thousands of lines of code by hundreds of programmers and predict its impact on the production environment? It can’t possibly work, which is why it doesn’t work.”

Dave Farley (co-author of Continuous Delivery):

“Complex approaches to gatekeeping, like ‘Change Approval Boards’ are negatively correlated with software quality, with the more complex compliance mechanisms around change associated with lower quality software.”

Gene Kim (co-author of The Phoenix Project, The DevOps Handbook):

“If every time we want to make a code change we have to send our engineers to scores of committee meetings in order to get permission to make our changes” — the countermeasure is to “create more loosely-coupled architecture so that changes can be made safely and with more autonomy.”

Charity Majors (co-founder of Honeycomb):

“The path to high performance is building better sociotechnical systems (fast deploys, observability, guard rails) that enable normal people to do great work.”

Nicole Forsgren (DORA co-founder):

“Knowledge is power, and you should give power to those who have the knowledge.”

6. The Proof: Organizations That Moved Beyond CABs

These are not startups. These are large, often regulated organizations that abandoned or dramatically reformed their CABs — and saw their performance soar.

ING Bank went from three separate CABs (planning, technical, deployment) to one lightweight deployment advisory board. The risk value per change decreased, and the ratio of incidents per change dropped significantly. Mark Heistek, IT specialist at ING, described the old CABs as being “just like an insurance company, always rejecting your first claim.”

Barclays Bank — a 328-year-old institution — under Jon Smart’s leadership, delivered 3x as much in 1/3 of the time with 23x fewer production incidents and the highest ever employee engagement scores. These results came from empowering teams, not from adding more approval gates.

Target went from deploying a few times per month to hundreds of times per day through their DevOps transformation.

Amazon deploys code every 11-12 seconds on average. Every engineer has full ownership of their services. No centralized CAB for routine deployments.

Netflix gives every engineer full access to the production environment from day one. Culture of accountability and continuous learning replaces approval gates.

Facebook/Meta moved to quasi-continuous “push from master” in 2017. An IEEE-published study found that continuous deployment did not inhibit productivity or quality even as the engineering team scaled by 20x and the codebase by 50x.

7. Even ITIL Agrees

Here is perhaps the most uncomfortable fact for CAB defenders: even ITIL itself — the framework that created the CAB — has moved on.

ITIL 4, released in 2019, makes sweeping changes:

1. Renamed from “Change Management” to “Change Enablement.” The name itself signals a philosophical shift from controlling to enabling.

2. CAB is no longer mandatory. ITIL 4 introduces the concept of a “Change Authority” that can be different for different types of changes and does not need to be a committee.

3. Aligned with Agile, DevOps, and CI/CD. ITIL 4 explicitly recognizes that change enablement should enable rather than restrict.

4. Automation encouraged. Unlike ITIL v3, which relied on manual approvals, ITIL 4 encourages automated change pipelines.

5. Standard changes are pre-approved. Low-risk, repeatable changes do not require individual approval.

Organizations that defend their CABs by citing ITIL compliance are, paradoxically, not compliant with the current version of ITIL. ITIL 4 supports exactly the approach that DORA research recommends: risk-based differentiation, automation, peer review, and team empowerment.

8. The Real Change Advisory Board: TDD and the Deployment Pipeline

If CABs don’t work, what does? The answer is not anarchy — it is engineering discipline.

Test-Driven Development creates a comprehensive safety net that replaces the need for external approval. A team practicing TDD does not need to ask permission to deploy. They can demonstrate that their change is safe. The test suite is the evidence. The pipeline is the judge.

The evidence is strong. A landmark study by Nagappan et al. at IBM and Microsoft found that TDD reduced pre-release defect density by 40-90% compared to similar projects not using TDD. DORA’s research found that test automation drives improved software stability, reduced team burnout, and lower deployment pain. DORA explicitly recommends that developers should “write unit tests before writing production code for all changes to the codebase” — a direct endorsement of TDD.

Robert C. Martin offers a powerful analogy: “I want you to think of TDD the way accountants think of dual entry bookkeeping.” Both are disciplines where every intention is entered in two places and must ultimately reconcile. CABs are bureaucratic discipline — a committee imposing external control. TDD is professional discipline — teams imposing rigorous standards on themselves. The latter is both more effective and more empowering.

Kent Beck, the creator of TDD: “Write tests until fear is transformed into boredom.” And: “Rather than apply minutes of suspect reasoning, we can just ask the computer by making the change and running the tests.”

The deployment pipeline — as described by Humble and Farley in Continuous Delivery — is the automated mechanism that enforces this discipline. Dave Farley argues it is “the perfect vehicle for compliance” and that he has seen organizations move “from taking weeks, sometimes months, to ensure that releases were ‘compliant,’ to generating genuinely compliant release candidates multiple times per day.”

Consider the properties:

PropertyDeployment PipelineCABConsistencyRuns the same checks every timeAttention varies by fatigue, time pressure, who’s in the roomSpeedFeedback in minutesFeedback in days or weeksObjectivityPass or fail — no politicsSusceptible to rubber-stamping, politics, bikesheddingComprehensivenessRuns thousands of testsReviews a change description documentAudit trailAutomatically records what was tested, when, and the resultMeeting minutes at best

What about business risk?

There is a legitimate distinction between technical validation (”are the tests passing?”) and business-risk approval (”is this safe to release now?”). A technically correct feature can still be a bad business decision to release on Black Friday.

Modern practices address this without a committee:

• Feature flags separate deployment from release. You deploy code to production without any user seeing it until a deliberate release decision is made.

• Progressive delivery (canary releases, percentage rollouts) manages blast radius mechanically, based on real production data rather than pre-deployment guesswork.

• Observability makes production behavior visible in real time, enabling detection and response in minutes.

DORA itself acknowledges this distinction. It recommends CABs transform “from gatekeeper to process architect,” focusing on strategic business decisions. The research does not say business risk assessment is unnecessary — it says the traditional CAB is the wrong mechanism for it.

For the rare change that genuinely requires business-risk assessment — a major infrastructure migration, a change with regulatory notification requirements — a lightweight, asynchronous, expert-driven review is the proportionate response. Not a weekly committee meeting. And certainly not for the 95%+ of changes that are standard code deployments.

9. The Compliance Theater Problem

Jez Humble cuts to the heart of it:

“Most change management is Change Management Theatre. It is not about making things better, it is about covering your ass when things go wrong.”

CABs create an illusion of control through several mechanisms:

The paper trail illusion. Change request forms create documentation. But documentation of a bad change does not prevent the bad change — it merely records that someone approved it.

The expert review illusion. CAB members are positioned as experts who will catch problems. But CAB members rarely have the codebase knowledge, context, or time to meaningfully evaluate individual changes. As Humble notes, heavyweight processes often result in forms being sent to approvers in another country who “don’t know anything about what they’re approving.”

The approval rate illusion. If a CAB approves 90%+ of changes — as the FCA found across UK financial services — the process is not filtering. It is rubber-stamping. A 90% approval rate means the CAB either lacks the ability to identify bad changes or lacks the courage to reject them. Either way, it is not performing its stated function.

The accountability illusion. The CAB provides someone to blame when things go wrong (”the CAB approved it”), but this diffused accountability reduces the incentive for teams to take ownership of quality.

The honest motivation behind many CABs is not risk reduction but liability protection. The real purpose is ensuring that when a failure occurs, someone can point to a signed-off change request form and say “I followed the process.” This protects individuals from blame but does nothing to prevent the failure itself.

10. The Psychology: Why CABs Refuse to Die

If the evidence is this clear, why do CABs persist? The answer lies not in technology or process, but in psychology, power, and identity.

Identity threat

When organizational changes threaten the value of professional roles, employees experience what researchers call identity threat — “an experience appraised as indicating potential harm to the value, meanings, or enactment of an identity” (Petriglieri, 2011). For CAB members who have spent years building their identity around being “the person who approves changes to production,” any suggestion that this role is unnecessary strikes at their core self-concept.

Research shows that identity threat leads to “exemplification behaviors” — performative actions designed to demonstrate one’s value. This maps directly to CAB members who, when threatened, intensify their gatekeeping — demanding more documentation, asking more questions, slowing down approvals — to demonstrate their indispensability.

Loss aversion and status quo bias

Kahneman and Tversky’s Prospect Theory established that losses loom larger than gains. Even when presented with DORA research showing that CABs make things worse, the cognitive calculus for those invested in the CAB is asymmetric: the potential loss of role, status, and identity weighs more heavily than the potential gain of organizational performance.

For someone who has championed the CAB for years, admitting it does not work also means admitting their past efforts were wasted (sunk cost), that they were wrong (cognitive dissonance), and that they will lose influence (control).

The ivory tower: authority without expertise

French and Raven’s framework on the Bases of Power is directly applicable. CAB members exercise legitimate power (authority from position) rather than expert power (authority from knowledge). The CAB inverts the natural power hierarchy: those with the least domain knowledge hold the most decision-making authority over the change.

Nicole Forsgren put it simply: “Knowledge is power, and you should give power to those who have the knowledge.” The CAB does the opposite.

The bus factor

CABs concentrate decision-making authority in a small group — often 3 to 7 people. When key members are unavailable, the entire deployment pipeline stalls. This is precisely the fragility that CABs claim to prevent. If the CAB chair is on holiday when a critical security patch needs to go out, the organization faces a choice: bypass the process or wait. Either way, the CAB has failed.

Theory X incarnate

Douglas McGregor’s Theory X assumes workers “dislike their work, avoid responsibility, and need constant direction.” Theory Y assumes work is natural and people will exercise self-direction when committed to objectives. CABs are a pure Theory X mechanism. They exist because of an implicit assumption that developers cannot be trusted to deploy responsibly.

McGregor identified the self-fulfilling prophecy: if you treat engineers as people who need external supervision, they internalize that belief and stop developing their own judgment. The CAB creates the dependency it then uses to justify its own existence.

The meta-irony

The Change Advisory Board resists change to itself. Bureaucratic inertia — “the tendency of bureaucratic organizations to perpetuate established procedures, resisting change irrespective of shifts in the environment” — ensures that every production incident becomes evidence that more oversight is needed, never that the oversight itself is contributing to the problem.

Conway’s Law compounds this: a CAB creates organizational roles (change manager, release coordinator, CAB secretary) that create constituencies whose job survival depends on the CAB’s continued existence. The process creates the roles. The roles defend the process.

David Graeber’s taxonomy of meaningless work applies directly. CAB reviewers who rubber-stamp changes they cannot evaluate are box tickers — performing roles “meant to make a firm appear to be doing something it’s not actually doing.” CABs that supervise change processes which would perform better without supervision are taskmasters — “people who supervise people who don’t need supervision.”

11. What Defenders Say — and Why They’re Mostly Wrong

It would be intellectually dishonest to pretend that no reasonable person defends CABs.

Gareth Davies of Lloyds Banking Group claims the CAB has kept them off the BBC front page. James Finister, an ITIL co-author, argues that “CABs stop the corridor conversations and the changes that bring an organization to its knees.” Stuart Rance, an ITIL practitioner, offers a nuanced middle ground: daily code deployments should bypass the CAB, but major infrastructure changes “like migrating cloud providers” may warrant broad stakeholder discussion.

These are fair points. But they conflate several different functions. The question is not whether organizations need risk assessment, cross-team coordination, or change visibility — they clearly do. The question is whether a weekly committee meeting is the best mechanism for delivering those functions. The DORA data says it is not.

CABs may serve functions beyond filtering: risk communication, stakeholder alignment, regulatory evidence, escalation. But a weekly meeting that adds up to 10 days of lead time, reduces deployment frequency, encourages larger batches, creates learned helplessness, and has no measurable effect on change fail rate is an expensive mechanism for functions that lighter alternatives can serve more effectively.

The TSB bank disaster of 2018 — where a botched IT migration left 2 million customers locked out — is frequently cited by CAB defenders. But the independent review revealed that TSB had governance structures; it failed to follow them. The root causes were inadequate testing, unrealistic timelines, and testing in production rather than pre-production. This is evidence for better engineering practices, not for more committees.

Conclusions

The evidence is not ambiguous. Across the largest study of software delivery performance ever conducted, across a regulator’s analysis of over 1 million production changes, across case studies from ING to Amazon to Barclays — the conclusion is consistent:

Change Advisory Boards do not do what they are supposed to do. They do not reduce change failure rates. They slow delivery. They force larger, riskier deployments. They erode team ownership. They create learned helplessness. They are compliance theater — providing the appearance of risk management without the substance.

The alternative is not a free-for-all. It is a rigorous, evidence-based approach to change management:

• Peer review by engineers who actually understand the code

• Test-Driven Development that validates every change at the atomic level

• Automated deployment pipelines that provide consistent, objective, comprehensive quality gates

• Feature flags and progressive delivery that separate deployment from release

• Observability and fast rollback that make production behavior visible and recoverable

• Risk-based differentiation that applies proportionate scrutiny — lightweight for routine changes, focused expert review for the rare high-risk change

• Team ownership — “you build it, you run it” — because the people closest to the code are the people best equipped to assess its risk

Even ITIL 4 — the framework that created the CAB — now supports this approach. It renamed “Change Management” to “Change Enablement,” made CABs optional, and aligned with Agile and DevOps practices.

The best change advisory board is not a room full of people skimming change request forms once a week. It is a deployment pipeline powered by automated tests written before the code, running thousands of checks in minutes, providing feedback a hundred times faster than any committee ever could — and backed by the professional discipline of engineers who own their code from commit to production.

The data has spoken. The question is whether your organization will listen — or whether the CAB will continue to do what CABs do best: resist change.

References

Books

1. Forsgren, N., Humble, J., Kim, G. (2018). Accelerate: The Science of Lean Software and DevOps. IT Revolution Press.

2. Humble, J., Farley, D. (2010). Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation. Addison-Wesley.

3. Kim, G., Humble, J., Debois, P., Willis, J., Forsgren, N. (2021). The DevOps Handbook, 2nd Edition. IT Revolution Press.

4. Kim, G., Behr, K., Spafford, G. (2013). The Phoenix Project. IT Revolution Press.

5. Skelton, M., Pais, M. (2019). Team Topologies. IT Revolution Press.

6. Schwartz, M. (2020). The (Delicate) Art of Bureaucracy. IT Revolution Press.

7. Smart, J. (2020). Sooner Safer Happier. IT Revolution Press.

8. Majors, C., Fong-Jones, L., Miranda, G. (2022). Observability Engineering. O’Reilly.

9. Beck, K. (2003). Test-Driven Development: By Example. Addison-Wesley.

10. Farley, D. (2022). Modern Software Engineering. Addison-Wesley.

11. Graeber, D. (2018). Bullshit Jobs: A Theory. Simon & Schuster.

Research Reports

12. DORA/Puppet. State of DevOps Reports (2014-2019). Multiple years of research on external approval and software delivery performance.

13. DORA/Google Cloud. (2019). Accelerate State of DevOps Report. Heavyweight approval 2.6x more likely in low performers.

14. DORA/Google Cloud. (2024). Accelerate State of DevOps Report. Continued emphasis on streamlining change approval.

15. Financial Conduct Authority. (2021). Implementing Technology Change. Multi-firm review. https://www.fca.org.uk/publications/multi-firm-reviews/implementing-technology-change

16. Nagappan, N., Maximilien, E.M., Bhat, T., Williams, L. (2008). “Realizing quality improvement through test driven development.” Empirical Software Engineering. https://www.microsoft.com/en-us/research/wp-content/uploads/2009/10/Realizing-Quality-Improvement-Through-Test-Driven-Development-Results-and-Experiences-of-Four-Industrial-Teams-nagappan_tdd.pdf

17. IEEE. (2016). “Continuous Deployment at Facebook and OANDA.” FSE 2016. https://ieeexplore.ieee.org/document/7883285/

Online Resources

18. DORA. “Streamlining Change Approval.” https://dora.dev/capabilities/streamlining-change-approval/

19. DORA. “Test Automation.” https://dora.dev/capabilities/test-automation/

20. Octopus Deploy. “Change Advisory Boards Don’t Work.” https://octopus.com/blog/change-advisory-boards-dont-work

21. ThinkingLabs. “Jez Humble: Continuous Delivery Sounds Great But It Won’t Work Here.” https://thinkinglabs.io/notes/2021/12/11/agiletd-continuous-delivery-sounds-but-it-wont-work-here-jez-humble.html

22. Latham & Watkins/JD Supra. “Implementing Technology Change — Successes and Pitfalls.” https://www.jdsupra.com/legalnews/implementing-technology-change-2874927/

23. Rance, S. “IT Change Management Doesn’t Always Need a CAB.” Optimal Service Management. https://www.optimalservicemanagement.com/blog/it-change-management-doesnt-always-need-a-cab/

24. InvGate. “Change Advisory Board Best Practices: 15+ Industry Leaders Weigh In.” https://blog.invgate.com/do-we-still-need-the-change-advisory-board

25. Martin, R.C. “Professionalism and TDD.” Clean Coder Blog. https://blog.cleancoder.com/uncle-bob/2014/05/02/ProfessionalismAndTDD.html

26. Farley, D. “Continuous Compliance.”

https://www.davefarley.net/?p=285

18. Farley, D. “Test Driven Development.”

https://www.davefarley.net/?p=220

18. Fowler, M. “Feature Toggles (aka Feature Flags).” https://martinfowler.com/articles/feature-toggles.html

19. Fowler, M. “Deployment Pipeline.” https://martinfowler.com/bliki/DeploymentPipeline.html

20. Humble, J. “Continuous Delivery and ITIL Change Management.” https://continuousdelivery.com/2010/11/continuous-delivery-and-itil-change-management/

ITIL 4 Sources

31. ITSM.tools. “Change Enablement in ITIL 4.” https://itsm.tools/change-enablement/

32. Joe The IT Guy. “What’s Changed with Change in ITIL 4?” https://www.joetheitguy.com/whats-changed-with-change-in-itil-4/

33. Beyond20. “Understanding Change Enablement Practice in ITIL 4.” https://www.beyond20.com/blog/understanding-change-enablement-practice-itil-4/

Case Studies

34. InfoQ. “ING Netherlands’ Transition to DevOps.” https://www.infoq.com/news/2014/06/ing-transtition-to-devops/

35. The Enterprisers Project. “Target CIO Explains How DevOps Took Root.” https://enterprisersproject.com/article/2017/1/target-cio-explains-how-devops-took-root-inside-retail-giant

36. InfoQ. “Benefits of Agile Transformation at Barclays.” https://www.infoq.com/news/2016/09/benefits-agile-barclays/

37. Barclays. “Insights: Jonathan Smart.” https://home.barclays/news/2018/02/insights-jonathan-smart/

38. Meta Engineering. “Rapid Release at Massive Scale.” https://engineering.fb.com/2017/08/31/web/rapid-release-at-massive-scale/

39. Panorama Consulting. “4 Lessons from the TSB Software Failure.” https://panorama-consulting.com/tsb-software-failure/

40. Bank of England. “TSB fined £48.65m for operational resilience failings.” https://www.bankofengland.co.uk/news/2022/december/tsb-fined-for-operational-resilience-failings

Psychological Frameworks

41. Petriglieri, J.L. (2011). “Under Threat: Responses to and the Consequences of Threats to Individuals’ Identities.” Academy of Management Review.

42. Kahneman, D., Tversky, A. (1979). “Prospect Theory: An Analysis of Decision under Risk.” Econometrica.

43. Seligman, M.E.P. (1972). Learned Helplessness. Annual Review of Medicine.

44. McGregor, D. (1960). The Human Side of Enterprise. McGraw-Hill.

45. Edmondson, A.C. (1999). “Psychological Safety and Learning Behavior in Work Teams.” Administrative Science Quarterly.

46. French, J.R.P., Raven, B. (1959). “The Bases of Social Power.” In Cartwright, D. (Ed.), Studies in Social Power.

47. Westrum, R. (1988). “Organizational and Inter-Organizational Thought.” World Bank Conference on Safety Control and Risk Management.

48. Corporate Rebels. “Learned Helplessness at Work.” https://www.corporate-rebels.com/blog/learned-helplessness-at-work

Here’s an uncomfortable truth about working with AI coding agents: we can’t review their output.

Not “we choose not to.” We can’t. Not meaningfully.

An AI agent can produce hundreds of lines of working code in seconds. A human reviewing that code line-by-line needs minutes, sometimes hours. The economics don’t work. And it’s getting worse — agents are getting faster, and the code they produce is getting more complex.

So what do teams actually do? They glance at the diff, run the existing tests, maybe squint at the architecture, and merge. We’ve replaced “trust but verify” with “trust and hope the CI is green.”

This isn’t sustainable. But I don’t think the answer is “just write more tests” either.

Where TDD and BDD Leave a Gap

Let me be clear: I’m not here to bury Test-Driven Development. TDD is one of the most important ideas in software engineering, and it works well with AI agents as well.

But TDD has a scope problem. It excels at specifying precise, deterministic behaviors — this input produces that output. It’s less natural for expressing the kind of requirements that live in the gaps: the relationship between multiple behaviors, the qualitative expectations (”the error message should be helpful, not cryptic”), the systemic properties (”this must hold even under high load”). These requirements are real. They matter to users. And they tend to live in the developer’s head, unwritten, because there’s no natural place to put them in a test file.

There’s also a phase problem. Before you can write a test, you need to know what to test. The design-time exploration — “what should this feature actually do, in all its edge cases?” — happens before TDD begins. It usually happens informally: in Slack, in a meeting, in someone’s head. It’s rarely captured.

BDD (Behavior-Driven Development) gets closer. Its Given/When/Then syntax forces you to think about behavior from the outside in, and its natural-language layer bridges the gap between business intent and executable code. But BDD’s formalism — specific frameworks (Cucumber, SpecFlow, Behave), step definitions, glue code — is also its strength: it’s what makes BDD scenarios executable and rerunnable. That rigor has a cost, though. It constrains how you express requirements. Some expectations are hard to fit into a three-line template.

What if we could keep the intent of BDD — specify behavior, then verify it — while trading some of its formalism for the flexibility to capture the full picture?

Enter Expectation-Driven Development

The idea is simple, maybe deceptively so:

1. Formulate expectations in plain text.

Not in a formal Given/When/Then template, though you can use that structure if it helps. Not in a programming language. In the same natural language you’d use to explain the feature to a colleague.

For example:

Expectation: Cart total calculation

When a user adds multiple items to their cart, the total should reflect the sum of all item prices multiplied by their quantities. If a discount code is applied, the discount should be calculated on the pre-tax subtotal, not on individual items. Tax is applied after the discount. If the cart is empty, the total should be zero, not an error.

Notice what this captures that a unit test wouldn’t: the relationship between discount and tax ordering, the empty cart edge case framed as an expectation about behavior (zero, not error), and the implicit requirement that this should work with “multiple items” — not just the two items in your test fixture.

You can go further:

Expectation: Race condition in concurrent booking

If two users attempt to book the last available slot at the same time, exactly one should succeed and receive a confirmation. The other should receive a clear rejection — not a timeout, not a double booking, not a corrupted state. This must hold even under high load.

Try writing that as a unit test. You’ll end up with a page of setup code that obscures the actual expectation. To be fair, nobody writes concurrency verification as a unit test — this would be an integration or stress test. But that’s the point. EDD’s job here isn’t to replace the concurrency test. It’s to make sure someone writes one, and that it tests the right thing. The expectation captures the requirement precisely; the test implements the verification.

2. Hand the expectations to the AI agent.

The agent implements the feature. It has the full context of your codebase, your expectations, and its own understanding of best practices.

3. Ask the agent to prove it.

This is the key step. Don’t just ask “did you implement it?” Ask: “Prove to me that each expectation is fulfilled. Show me specific scenarios with concrete inputs and the outputs the system produces.”

The agent should respond with evidence like:

Evidence for: Cart total calculation

Scenario 1: Added “Widget” ($10.00, qty 2) and “Gadget” ($25.00, qty 1). Subtotal: $45.00. Applied discount code “SAVE10” (10% off). Discount: -$4.50. After discount: $40.50. Tax (8%): $3.24. Total: $43.74.

Scenario 2: Empty cart. GET /api/cart/total returns { "total": 0.00, "items": [] } with HTTP 200. No error thrown.

Scenario 3: Added “Widget” ($10.00, qty 1). No discount. Tax (8%): $0.80. Total: $10.80. Tax applied to full subtotal, not discounted amount (no discount to apply).

4. Evaluate, challenge, iterate.

Read the evidence critically. Does it actually cover the expectation? Did the agent dodge the hard part? Are the numbers right? Push back:

“You showed the discount applied to the subtotal, but you didn’t show what happens if someone tries to apply two discount codes. What happens then?”

The agent revises. New evidence is produced. You iterate until convergence.

5. The evidence becomes the documentation.

When you’re done, you have something valuable: a set of expectations paired with concrete proof that the system meets them. This isn’t a test suite that requires a framework to run. It’s not a wiki page that was outdated the day it was written. It’s a living record of what the system does and why, backed by specific examples.

The Workflow, Visualized

Human: Formulates expectations (plain text)
  │
  ▼
AI Agent: Implements the feature
  │
  ▼
Human: "Prove it meets the expectations"
  │
  ▼
AI Agent: Produces evidence (concrete scenarios, inputs, outputs)
  │
  ▼
Human: Reviews evidence ──── Satisfied? ──── YES ──→ Document & ship
  │
  NO
  │
  ▼
Human: Challenges, adds expectations
  │
  └──────── Loop back to AI Agent ────────────┘

This Isn’t a New Idea (And That’s a Good Thing)

If you’ve been in software long enough, you’re probably thinking: “This sounds like Specification by Example.” You’re right. It does.

EDD stands on the shoulders of a long lineage of ideas that share the same core insight — that concrete examples in human-readable form are the best way to specify and verify software behavior:

• Specification by Example (Gojko Adzic, 2011) — uses concrete examples collaboratively authored by the team as both specifications and tests. The bible for this way of thinking.

• FIT / FitNesse (Ward Cunningham, ~2002) — executable acceptance tables written in near-plain-English, verified automatically against the system.

• Concordion — specifications written in natural language that become executable through instrumentation.

• Design by Contract (Bertrand Meyer, Eiffel) — preconditions, postconditions, and invariants as formal specifications embedded in code.

• Property-based testing (QuickCheck and descendants) — addresses the “not just the two items in your test fixture” problem by generating many inputs from declared properties.

So what’s actually new here? Not the idea of specifying behavior in natural language. Not the idea of verifying with concrete examples. What’s new is the execution context.

In Specification by Example, a human team collaborates to write examples, then a developer writes glue code (step definitions) to make them executable. The bottleneck is the glue code — it’s tedious, it breaks when the code changes, and it requires a framework.

In EDD, the LLM is the glue code. It interprets natural-language expectations directly, without step definitions, without a framework, without the ceremony. And the verification loop is conversational — you challenge, the agent responds, you push back — rather than pass/fail binary.

That’s a meaningful difference. But I want to be honest that it’s an evolutionary one, not a revolutionary one. If you’ve read Adzic’s work, you’ll feel at home here. If you haven’t, go read it — it will make you better at EDD.

There’s one more thing EDD inherits from Specification by Example: the question of who writes the expectations. In the Specification by Example tradition, examples are collaboratively authored in “specification workshops” — conversations between developers, testers, and business stakeholders. EDD as I’ve described it is a solo workflow: one human, one agent. But most software is built by teams. If different team members write expectations for the same feature, they may express different — or contradictory — assumptions. If only one person writes them, you’ve concentrated a single point of failure in that person’s understanding. I don’t have a neat answer for this yet. My instinct is that expectations should be written collaboratively, then the conversation with the agent happens individually. But this is an open question.

Why This Might Actually Work

It plays to human strengths. Humans are better at judging than producing. We’re excellent critics and mediocre typists. EDD lets us do what we do best: specify intent, evaluate outcomes, spot what’s missing.

It plays to AI strengths. AI agents are fast, tireless implementers that can generate both code and evidence at scale. The bottleneck was never “can the AI write the code?” It was “can we trust the code the AI wrote?” EDD creates a structured trust-building process.

Natural language captures what formal tests can’t. “The error message should be helpful, not cryptic.” “The response time should feel instant for typical queries.” “The fallback behavior should be graceful, not surprising.” These are real requirements that matter to users. They’re nearly impossible to express in assert statements, but an LLM can interpret them. A caveat: for subjective qualities like “helpful” or “graceful,” the LLM will tend to judge its own output favorably — it’s the fox-guarding-the-henhouse problem again. For these expectations, the human’s judgment in Step 4 becomes especially critical. Don’t outsource taste.

It forces you to think before coding — the best part of TDD, without the ceremony. The discipline of TDD was never really about the tests. It was about the act of specifying behavior before implementation. EDD preserves that discipline. You still think first. You just express your thinking in a more natural medium.

The evidence trail creates accountability. Every expectation has a proof artifact. Six months from now, when someone asks “does the system handle concurrent bookings correctly?”, you don’t grep through test files. You read the expectation and its evidence.

Why This Might Not Work (The Honest Part)

I believe in this idea, but I’d be dishonest if I didn’t confront its weaknesses head-on. There are real problems here.

The Fox Guarding the Henhouse (And the Deeper Problem Beneath It)

This is the big one, and it has two layers.

Layer 1: Bias. You’re asking the same AI that wrote the code to produce evidence that the code works. That’s like asking a student to both take the exam and grade it. The AI has every structural incentive to produce evidence that confirms its implementation. If it made a subtle error in the discount calculation, it might generate scenarios that avoid the precise inputs that would reveal the bug. Not maliciously. Just because the same reasoning flaw that caused the bug will also cause it to overlook the bug in its evidence.

Layer 2: Execution vs. narration. This is the deeper problem, and it’s one we need to confront directly. When the agent produces “evidence,” what actually happened? There are two very different possibilities:

• Executed evidence: The agent actually ran the code — made an API call, executed a function, queried the database — and is showing you real output from a real system.

• Generative evidence: The agent described what it believes would happen, based on its understanding of the code it wrote. It narrated a plausible verification without executing anything.

These are not the same thing. Executed evidence is evidence. Generative evidence is a second assertion by the same entity that made the first assertion. It’s the difference between “I tested it and here are the results” and “I’m pretty sure it would work like this.”

EDD requires executed evidence. If the agent can’t run the code and show you real outputs, you don’t have verification — you have a shared hallucination dressed up in scenario format. This means EDD depends on AI agents with tool use: the ability to execute code, call APIs, run scripts, and capture actual output. Fortunately, this is where agents are headed — modern coding agents already have shell access, can run test suites, and can interact with running systems. But you must insist on it. When reviewing evidence, ask: “Did you actually run this, or are you telling me what you think would happen?” If the agent can’t answer that clearly, the evidence is worthless.

But let’s be honest: not all code is executable in an agent loop. In practice, evidence falls into three categories:

• Directly executable: Functions, APIs, scripts, database queries. The agent can run these and show real output. This is the gold standard.

• Partially verifiable: Infrastructure code (Terraform plan output), build configurations (dry runs), schema migrations (against a test database). The agent can’t deploy to production, but it can show you what would happen. This is weaker but still useful — a Terraform plan is better than a guess.

• Not executable in the loop: UI rendering, production-only behavior, third-party API integrations with rate limits or authentication, code that requires manual user interaction. Here, you’re back to generative evidence whether you like it or not.

For that third category, the mitigations below become critical, and you should be clear-eyed that your confidence is lower. If most of your expectations fall into category three, EDD’s value proposition degrades — and you should invest more in the “Stabilize” step (converting expectations to automated tests that can run in CI).

Mitigations for both layers:

• Be an adversarial reviewer. Your job isn’t to passively receive evidence. It’s to actively challenge it. Run adversarial scenarios. Ask “what about...?” questions. Spot-check the numbers manually.

• Demand execution receipts. Ask the agent to show the actual command it ran and the raw output. Not a summary. The output.

• Use a different agent to audit. Have a second AI agent (or a different model) independently verify the evidence against the code. The fox can guard the henhouse if there’s a different fox checking the first fox’s work.

• Spot-check yourself. For critical expectations, pick one scenario and run it manually. If the agent’s evidence matches reality for the one you checked, you have higher confidence in the rest.

The Reproducibility Question

Here’s a hard question: are the expectations rerunnable?

If I come back next week, after the code has changed, can I re-verify the expectations? If yes — how? If you’re asking an AI to re-run the evidence each time, you’re relying on LLM interpretation, which is non-deterministic. The same expectation might produce different evidence on different runs. The same model might interpret an ambiguous expectation differently after an update.

If the expectations aren’t rerunnable, then what you have is a snapshot, not a safety net. Documentation, not regression protection.

Mitigation: EDD should complement automated tests, not replace them. The expectations drive the initial implementation and verification. But the critical paths should also be captured in traditional automated tests for regression. Think of expectations as the design-time validation tool, and automated tests as the runtime safety net. The expectations are the “why.” The tests are the “what, forever.”

There’s a related versioning problem worth flagging. If you ship feature v1 with its evidence, then modify the feature in v2, the v1 evidence is now potentially misleading. Do you re-run evidence for every change? If so, you’re paying the EDD cost on every iteration. If not, the documentation rots like any other documentation. Automated tests don’t have this problem — they fail loudly when the code changes in ways they don’t expect. EDD evidence is silent when it becomes stale. This is another reason the “Stabilize” step matters: the automated tests are your regression alarm. The expectations are your design-time conversation.

Natural Language Ambiguity

I praised the freedom of natural language earlier. But freedom has a cost. Consider:

“The system should handle large uploads efficiently.”

What’s large? 10MB? 10GB? What’s efficiently? Under 5 seconds? Without running out of memory? Without blocking other requests?

In BDD, the formalism forces you to be specific: Given a file of 500MB / When the user uploads it / Then the upload completes within 30 seconds. The rigidity is a feature. It prevents hand-waving.

Mitigation: Write expectations as specifically as you can. The freedom of natural language doesn’t mean you should be vague — it means you can be specific in ways that formal syntax doesn’t allow. Instead of “handle large uploads efficiently,” write “a 500MB upload should complete without timeout and without consuming more than 2x the file size in memory. A 5GB upload should use streaming and never hold the full file in memory.” Still natural language. But precise.

The Evidence Scalability Problem

If you have 5 expectations, you can carefully review 5 sets of evidence. If you have 50, you’ll start skimming. If you have 200, you’ll rubber-stamp.

And yet, more expectations means better coverage. There’s a tension between thoroughness and human attention span.

Mitigation: Prioritize. Not all expectations are equal. Some protect critical business logic. Some cover edge cases that matter but don’t need forensic review. Categorize expectations by risk and allocate your review attention accordingly.

EDD vs. TDD vs. BDD: Not a Replacement, a Complement

Let me be blunt: if you read this and think “great, I can stop writing tests,” you’ve missed the point.

TDD speaks to developers, in code. It’s always executable, gives you strong regression protection, and works well as a design-time thinking tool. But it captures limited nuance — an assert can only say so much — and it wasn’t designed with AI agents in mind (though “make these tests pass” works surprisingly well).

BDD speaks to the whole team, in structured natural language. It’s executable via step definitions, gives you strong regression protection, and captures more nuance than raw code. But it still constrains how you express requirements, and it wasn’t designed for AI agents either.

EDD speaks to the human-AI pair, in free-form natural language. The expectations themselves aren’t executable — they’re text — but the protocol demands execution in Step 3. Regression protection is weak without automation (which is why the Stabilize step matters). Where EDD wins is nuance: natural language can express the qualitative, relational, and systemic requirements that formal test syntax struggles with. And it’s designed from the ground up for the AI-agent workflow.

The three aren’t competitors. They’re layers. EDD is the conversation layer between human intent and AI implementation — where you say what you mean, in all its messy, nuanced, edge-case-laden glory. Then TDD or BDD takes over for the parts that need to be deterministic and repeatable.

A Practical Protocol

If you want to try EDD today, here’s a concrete protocol:

Step 1: Write expectations before touching code

Spend 15 minutes writing expectations for the feature. Be specific. Cover:

• The happy path

• Edge cases you’ve been burned by before

• Non-functional requirements (performance, error handling, security)

• Behaviors that should explicitly not happen

How big should an expectation be? Think “one behavior you’d explain in a single breath to a colleague.” The cart total calculation example is about right — it covers one coherent concern (pricing math) with its key edge cases. If you find yourself writing a page-long expectation, you’re probably bundling multiple concerns. Split them. If you find yourself writing a one-liner with no edge cases, you’re probably being too vague. Expand it.

Step 2: Hand expectations to the AI agent with your codebase

Give the agent your expectations and let it implement. Don’t hover. Let it work.

Step 3: Request executed evidence

Prompt: “For each expectation, show me concrete evidence that the system fulfills it. Actually run the code — execute the function, call the API, run the script — and show me the real inputs and outputs. Don’t describe what you think would happen; show me what actually happened. If an expectation cannot be fulfilled, explain why.”

The word “actually” is doing heavy lifting here. You want execution receipts, not narration.

Step 4: Review adversarially

For each piece of evidence:

• Do the numbers add up?

• Did the agent test the edge case or dodge it?

• What input would break this?

• Is there a scenario the expectation didn’t cover that it should?

Step 5: Challenge and iterate

Add new expectations based on what you discover. Tighten vague ones. Ask “what if?” until you’re satisfied.

Step 6: Stabilize

For critical paths, convert the expectations and evidence into automated tests (unit, integration, or e2e). The evidence gives you the test cases for free — you just need to make them executable and deterministic.

Step 7: Archive

The final set of expectations + evidence becomes your feature documentation. It answers “what does this do?” and “how do we know?” in one artifact.

A Disclaimer, and the Deeper Point

I should be upfront: I haven’t battle-tested EDD on a real project. This is a proposed framework, not a field report. I’m sharing it because I think the underlying problem — how do we validate AI-generated code at the speed AI generates it? — is urgent enough that we should be thinking out loud about solutions, even imperfect ones. If you try this and it works, I want to hear about it. If you try it and it falls apart, I want to hear about that even more.

That said, I think EDD points at something deeper than a specific technique. It reflects a shift in the role of the human developer.

In the pre-AI world, we were authors. We wrote the code. We wrote the tests. We wrote the documentation. Our value was in production.

In the AI-agent world, we’re becoming editors. We specify intent. We evaluate output. We challenge evidence. Our value is in judgment.

Authors need tools for writing — IDEs, compilers, debuggers. Editors need tools for evaluating — ways to express what they want, inspect what they got, and close the gap between the two. TDD is an author’s tool. EDD is an attempt at an editor’s tool.

We’re early. The tooling isn’t there yet. The methodology needs pressure-testing by real teams on real projects. But the direction feels right: humans specifying intent, machines producing implementations, and a structured conversation in between to build justified confidence that the two actually match.

The question isn’t whether AI agents will write most of our code. They already do. The question is whether we’ll find a way to stay confident that the code does what we intended. Expectation-Driven Development is one bet on how we get there. I’m looking for others.

“Inspection is too late. The quality, good or bad, is already in the product. Cease dependence on inspection to achieve quality. Eliminate the need for inspection on a mass basis by building quality into the product in the first place.”

-- W. Edwards Deming, Out of the Crisis (1982)

Abstract

Pull requests have become the default code review mechanism in software teams everywhere. But the pull request was invented for open source projects, where strangers contribute code to repositories maintained by people who don’t know them. When private teams of trusted colleagues adopt the same process, they import a model designed for low trust into an environment that should operate on high trust. The result is a post-development inspection system that the evidence says catches few bugs, introduces enormous waiting time, incentivises large batches, and fractures teams into isolated individuals. Academic research, large-scale industry data from DORA, and a growing practitioner consensus all point in the same direction: there are better ways to build quality into software than inspecting it after the fact. This article examines the evidence and proposes an alternative — T*D, the union of Test-Driven Development, Trunk-Based Development, and Team-focused Development — as a path from slow, inspection-heavy workflows to fast-flowing teams that produce high-quality, safe systems.

TL;DR

• Pull requests were designed for open source contributions from untrusted strangers. Applying them to trusted teams is a category error.

• Peer-reviewed research shows code review’s primary value is knowledge transfer, not bug detection. Less than 15% of review comments relate to actual bugs.

• Async PR workflows mean your code spends 86-99% of its lead time waiting. One organisation spent 130,000 hours in a single year waiting on PRs that received zero comments.

• DORA research across 36,000+ professionals shows trunk-based development correlates with dramatically higher software delivery performance, and faster code reviews alone improve performance by 50%.

• The alternative is T*D: Test-Driven Development (build quality in), Trunk-Based Development (integrate continuously), and Team-focused Development (review during creation, not after).

• The transition is gradual: optimise PRs first, adopt Ship/Show/Ask, then move to pairing and trunk-based development as trust and automation mature.

Deming wrote those words about manufacturing in 1982, but they describe what happens in most software teams today with uncanny precision. A developer writes code in isolation, on a branch, for hours or days. Then they open a pull request. The code sits in a queue. Someone eventually looks at it, leaves a few comments -- mostly about naming and formatting -- clicks approve, and the code merges. The whole process feels thorough. It feels like quality control. But the evidence suggests it is mostly theatre.

This article is not against code review. Code review has real value -- but that value is knowledge transfer and shared understanding, not catching bugs. And a blocking, asynchronous pull request is one of the worst possible mechanisms for achieving it.

What follows is a synthesis of peer-reviewed academic research, large-scale industry data, and practitioner experience spanning two decades. The picture that emerges is consistent and, for many teams, uncomfortable: the way most organisations practice code review is an expensive ritual that slows delivery, encourages large batches, fractures team cohesion, and produces a false sense of security. Better alternatives exist, and they aren’t theoretical -- they are practised successfully by high-performing teams around the world.

A Solution Designed for Strangers

Pull requests have a specific origin story, and understanding it explains why they are a poor fit for most teams.

Linus Torvalds created the git-request-pull command in 2005, shortly after releasing Git. Before that, open source projects used email-based patch workflows: contributors mailed diffs to a mailing list, and the maintainer decided whether to apply them. In 2008, GitHub launched the web-based pull request, making it dramatically easier to accept contributions from the outside world -- from people the maintainers did not know.

This was a genuine innovation for open source. The pull request was designed as a gatekeeper mechanism for untrusted contributors submitting code to repositories they don’t own. It solved a real problem: how do you safely accept code from strangers?

But then something happened. Git became the dominant version control system. GitHub became the dominant platform. And teams everywhere adopted the pull request as their default workflow -- not because they had evaluated it against alternatives, but because it was there and everyone else was using it. As Martin Fowler observed: “I suspect that since pull requests are so popular, a lot of teams are using them by default when they would do better without them.”

The category error should be obvious. In an open source project, you are reviewing code from someone you may never have met, working in a codebase they may not fully understand, with no shared context about the team’s conventions or direction. A gatekeeper model makes sense. In a private team, you are reviewing code from a colleague who sits in the same stand-up, shares the same goals, and (presumably) has been hired precisely because you trust their competence. Yet the process is the same.

Thierry de Pauw puts it bluntly: “Pull requests are designed to make it easier to accept contributions from the outside world, from untrusted people we do not know about.” When your team adopts this model internally, you are importing a trust assumption that does not match your reality.

Even in open source, pull requests cause friction. Contributions sit unreviewed for weeks or months. Maintainer burnout is well documented. The process works better than emailing patches, but it is hardly frictionless. And the friction that is merely inconvenient in open source becomes genuinely destructive in a team that needs to integrate and deploy multiple times a day.

What Code Review Actually Finds

If you ask developers why they do code review, the most common answer is: to catch bugs. The evidence says this is largely a myth.

The most significant data comes from Microsoft Research. A 2015 IEEE study titled “Code Reviews Do Not Find Bugs: How the Current Code Review Best Practice Slows Us Down” found that only a very small percentage of code review comments had anything to do with bugs. Most were about structural issues and style. A landmark 2013 paper by Bacchelli and Bird -- “Expectations, Outcomes, and Challenges of Modern Code Review” -- studied code review at Microsoft through observation, interviews, and surveys and reached the same conclusion: while finding defects remains the main motivation for review, reviews are less about defects than expected. Less than 15% of issues discussed in code reviews relate directly to bugs. The primary benefits are knowledge transfer, increased team awareness, and the creation of alternative solutions.

A separate Microsoft Research study analysed 1.5 million review comments across five projects. The more files in a change, the lower the proportion of useful comments. Developers spent on average six hours per week reviewing others’ changes. Reviews containing more than 20 files were already too big for effective review.

Large-scale industry data tells the same story. One major technology company, after examining nine million code reviews internally, cited knowledge transfer as the primary source of code-review ROI, not bug finding. Up to 75% of code review comments affect software evolvability and maintainability rather than functionality.

None of this means code review is worthless. It means its value lies in a different place than most people assume. If your organisation justifies pull requests primarily as a bug-catching mechanism, you are optimising for a benefit that peer-reviewed research says is marginal. The knowledge transfer benefit is real -- but a blocking asynchronous queue is one of the worst ways to achieve it.

The Staggering Cost of Waiting

Here is a calculation that should disturb any engineering leader.

If a code change takes 10 minutes to make but waits 1 hour for review, it is waiting for 86% of its total lead time. If review takes 4 hours: 96% waiting. If review takes one working day -- which is common -- the code spends 99% of its existence waiting for a human to look at it.

Martin Fowler cites a client that spent 130,000 hours in 2020 waiting for 7,000 pull requests that had no comments. Ninety-one percent of their PRs received no comments at all. The vast majority were rubber-stamped without substantive review, yet the process still imposed enormous delay.

The damage compounds through context switching. Research shows that developers wait an average of four days for a pull request review, that 86% of pull requests are handled under context-switching conditions, and that context-switching pull requests are 223% slower than non-context-switching ones. When a developer opens a PR and moves on to something else, rebuilding the mental context of the original work takes 30-60 minutes -- if they rebuild it at all.

Don Reinertsen’s The Principles of Product Development Flow provides the theoretical lens: batch size is an economic tradeoff between holding cost and transaction cost, and halving batch sizes halves queues and halves cycle time. Pull requests create a perverse incentive toward larger batches: because the transaction cost of getting a review is high (waiting, context switching, reviewer availability), developers batch more changes into each PR to amortise the review cost. Larger PRs take longer to review, reviews are less effective on large changes, and the cycle reinforces itself.

Charity Majors frames the cost financially. A 6-person team requiring days to deploy would need 24 people to match the output of a team deploying continuously -- roughly $3.6 million in unnecessary salary costs. A 10-person team shipping weekly would need 80 people: $14 million in waste. Her target: “Any merge triggers automatic deploy to production, completed in 15 minutes or less with no human intervention.”

Async Reviews: The Vicious Cycle

Dragan Stepanovic examined async code review workflows across more than 30 active repositories and observed a counter-intuitive pattern: teams using small PRs with async code reviews tended to have lower throughput than teams using large PRs. The reason is a feedback loop. Waiting for review leads to high work-in-progress (developers pull in new work while waiting). High WIP means less availability for reviewing. Less availability means more async handoffs. More async handoffs mean more waiting.

Instead of forming one team, developers become what Stepanovic calls “N teams of one person, often with different engineering cultures and coding practices.” The isolation is the opposite of what high-performing teams need.

Chelsea Troy identifies the root cause as insufficient shared context. Asynchronous code review places a massive demand on the reviewer’s time because they lack the context the author built over hours or days. Pairing is more efficient precisely because the context transfer is immediate -- the work of understanding and reviewing happens in a condensed amount of time relative to the amount of work being done.

A caveat is warranted: these are practitioner observations, not controlled experiments. The causal mechanism is plausible but not empirically proven. It is possible that teams with lower throughput default to async review for other reasons. But the observations are consistent across multiple independent practitioners, and the logic of WIP accumulation under queuing theory is well established.

What DORA Tells Us

If the practitioner arguments are directional, the DORA data provides the scale.

Accelerate, by Nicole Forsgren, Jez Humble, and Gene Kim, is based on four years of research across 23,000 surveys from over 2,000 organisations. Their findings on trunk-based development are unambiguous: developing off trunk rather than long-lived feature branches correlated with higher delivery performance. High-performing teams had fewer than three active branches at any time, branches lasted less than a day, and there were no code freezes or stabilisation periods.

The DORA capabilities framework identifies trunk-based development as a key technical capability. Elite performers who meet reliability targets are 2.3 times more likely to use trunk-based development. Crucially, the framework explicitly calls out “heavyweight code review processes” as a barrier -- they push developers toward larger batches and delay merges.

The 2023 State of DevOps Report, based on 36,000+ professionals worldwide, found that accelerating the code review process alone can lead to a 50% improvement in software delivery performance. Note what this says: the problem is not review itself, but the speed of review. Make review instant -- through pairing, rapid turnaround, or automation -- and you keep the benefit without the cost.

An important caveat: the DORA research is correlational, not causal. It shows that high-performing teams tend to use trunk-based development, not that trunk-based development causes high performance. Teams that adopt TBD may also be better-funded, more skilled, or have stronger engineering cultures. But the consistency of the finding across years and across tens of thousands of respondents makes it the strongest industry evidence available.

The Minimum CD initiative, co-created by Jez Humble, is blunt about the implication: daily integration to trunk is non-negotiable. If your team is not integrating to trunk daily, you are not doing continuous integration. Pull requests, as typically practised, fail this test.

A Growing Practitioner Consensus

ThoughtWorks placed “Peer review equals pull request” in the HOLD ring of their Technology Radar in April 2021 -- meaning “proceed with caution.” They noted that PRs create significant team bottlenecks, degrade review quality as overloaded reviewers begin rubber-stamping, and in one case, a client’s regulatory audit found that pull requests did not satisfy compliance because there was no evidence the code was actually read.

Kief Morris, also of ThoughtWorks and author of Infrastructure as Code, argues that pull requests add overhead designed for low-trust situations and that pull requests are not continuous integration -- CI is an alternative to pull requests, not a complement.

Dave Farley, co-author of Continuous Delivery, argues that pull requests are an artifact of branch-based development which deliberately isolates changes from mainline -- the opposite of what CI requires.

Jason Gorman asks the most piercing question: “Ask not so much ‘How do we do Pull Requests?’ but rather ‘Why do we need to do Pull Requests?’” His answer: pull requests are a symptom of low trust, not a solution for low quality. Teams should address the root causes -- skills development, professional standards, pair programming -- rather than institutionalising slow inspection.

And perhaps the most quotable line comes from Jessica Kerr, amplified by Kent Beck: “Pull requests are an improvement on working alone. But not on working together.”

T*D: Build Quality In

If pull requests are Deming’s inspection, what is the alternative? What does it look like to build quality into the process itself?

The answer is a union of three practices I’ll call T*D -- a deliberate echo of TDD, because all three components share the same initials and the same philosophy of shifting quality left.

Test-Driven Development

Write a failing test. Write the minimum code to pass it. Refactor. Repeat.

Microsoft and IBM studies found that TDD reduced pre-release defect density by 40-90% compared to projects not using it, at a cost of 15-35% more development time. Kent Beck’s insight was that automated testing replaces fear with confidence: when stress increases, developers run tests rather than skipping them.

TDD’s role in replacing PRs is straightforward: when every line of code is written to satisfy a test, the code has already been verified before it reaches anyone. The automated safety net catches regressions immediately. You do not need a human gatekeeper to tell you whether the code works -- the tests do.

Trunk-Based Development

All developers merge to the main branch at least daily, working in small batches. Feature flags manage incomplete work.

The evidence, covered extensively above, is unambiguous: trunk-based development is correlated with higher delivery performance across every DORA metric. Paul Hammant’s trunkbaseddevelopment.com establishes that the ideal branch duration is one day maximum, with the smallest pieces being a quarter of a day. Feature flags eliminate the argument that “we need branches because features aren’t complete.”

TBD’s role in replacing PRs is structural: when changes are tiny (hours, not days), the review burden is minimal and can be handled through pairing or post-commit review. There is nothing to gate because there is nothing large enough to fear.

Team-Focused Development

Two or more developers work on the same code at the same time, providing continuous real-time review.

This is the component with the most nuance. The peer-reviewed evidence for pair programming’s quality benefits is substantial. Williams et al. (2000) found pairs produced 15% fewer defects. Hannay et al.’s 2009 meta-analysis -- the most comprehensive to date -- found a “small significant positive overall effect on quality”, strongest on complex tasks. Arisholm et al.’s 2007 study of 295 professional developers found a 48% increase in correct solutions on complex systems. Junior developers showed the strongest benefit: 149% improvement on complex tasks.

Most critically for our argument, Muller and Tichy (2005) directly compared pair programming to solo programming with peer review. Their central finding: pairs and solo developers with peer review achieve equivalent cost when quality is held constant. The review phase for solo developers adds enough overhead to match the cost of pairing. This means pair programming does not cost more than solo-plus-review -- it simply moves the quality assurance from after development to during it.

Ensemble (mob) programming extends this: the whole team works on the same thing, at the same time, on the same computer. Review happens instantly, after every line. The academic evidence for mob programming is still preliminary, but the practitioner consensus is strong: when the whole team creates together, there is no need for post-hoc inspection.

An honest caveat: no peer-reviewed study has directly proven that teams can safely eliminate post-hoc review when practising pair programming. The evidence shows cost-equivalence and comparable quality, not that one fully replaces the other. The argument that pairing eliminates the need for PRs is practitioner consensus extending academic evidence -- well-reasoned, but not empirically proven.

The Synthesis

When combined, these three practices create a system where quality is built in at every stage:

• TDD means every line of code is verified by an automated test before it exists

• TBD means changes are tiny, integrated frequently, and never far from mainline

• TFD means human review happens continuously, during creation, not after

Fear of bugs? TDD catches them at creation time. Need for review? Pair programming reviews continuously. Integration risk? TBD integrates frequently in small batches. Knowledge sharing? Co-creation shares knowledge in real time.

The result is what Dave Farley describes as “Extreme Programming practices of ensemble programming and continuous code review that eliminate all waiting and waste.”

The Quality Journey: From Fear to Flow

I want to address the hardest question honestly: what if your team genuinely isn’t ready to drop pull requests?

This is a real concern. Jason Gorman acknowledges that the need for PRs often indicates a real skills gap. Thierry de Pauw argues that distrust-driven PR adoption signals deeper problems -- legacy code comprehension issues, blame cultures, or absent trust in engineers. No process fixes fundamental cultural dysfunction. But the answer is not to permanently institutionalise a slow inspection process. It is to address the root causes directly while gradually reducing dependence on gating review.

Here is a transition path, drawn from practitioner experience:

Stage 1: Optimise PRs. If you’re not ready to eliminate them, make them less harmful. Enforce smaller PRs (under 200 lines). Set review SLAs (under 4 hours). Automate all style and lint checks. Reduce required approvers to one. This alone can dramatically improve flow.

Stage 2: Ship/Show/Ask. Rouan Wilsenach’s model, published on martinfowler.com, categorises changes into three types. Ship: merge directly to mainline without review -- for routine changes where the developer is confident. Show: open a PR, merge immediately without waiting, and let review happen post-merge. Ask: open a PR and wait for discussion -- for complex or uncertain changes. This reduces the proportion of work that needs blocking review and starts building the muscle of trust.

Stage 3: Pair programming plus trunk-based development. Pair on all production code. Merge to trunk multiple times daily. Automated tests run on every commit. Post-commit review covers anything that wasn’t paired on.

Stage 4: Ensemble programming plus trunk-based development. The full team collaborates on code. Continuous review during development. Direct trunk commits. Feature flags for incomplete work.

Stage 5: Full T*D. All three practices operating together. Quality built in through TDD. Continuous integration through TBD. Continuous review through pair or ensemble. A fast automated deployment pipeline. Feature flags managing incomplete work. This is not a fantasy -- it is how the highest-performing teams in the industry operate.

Thierry de Pauw documents a specific intermediate approach: non-blocking continuous code review. Reviews happen on mainline after merging, on a per-feature level. Teams add a “To Review” column to their board. Developers check for pending reviews before starting new work. No hierarchical requirements -- seniors review juniors, juniors review seniors. Unreviewed code may reach production, but only if it passed automated testing. Notably, internal auditors in one organisation found this model superior to traditional checkbox-based compliance.

The key insight from Charity Majors is that speed and safety are not trade-offs -- speed IS safety. “Ship a single changeset by a single dev at a time, making it easy to isolate the owner of any problem, preventing the blast radius from expanding, and making it easy to fix while the intended effects of the code are fresh in their mind.”

Trust Is a Gradient

One final nuance. The argument that PRs are designed for “untrusted strangers” and therefore wrong for “trusted teams” implies a binary. In reality, trust is a gradient.

A small, co-located team of five people who pair daily has very high trust. The case against async PRs is strongest here. A large organisation with hundreds of engineers across multiple teams has lower cross-team trust. Within a team, pairing and trunk-based development may be ideal; across team boundaries, lightweight PRs with fast SLAs may still serve a purpose. For inner-source or platform teams that accept contributions from semi-external developers, a gatekeeper model may remain appropriate -- but even then, the goal should be fast, non-blocking review, not multi-day async queues.

For distributed teams with significant timezone overlap, pair programming via screen sharing works during overlap windows, with non-blocking post-commit review for work done outside overlap. For teams with minimal overlap, async PR workflows may be the least-bad option, but should be optimised aggressively: small changes, 4-hour review SLAs, automated quality gates, and a single required reviewer.

The antipattern diagnosis applies most strongly to same-team, same-context work where async blocking PRs replace trust that should already exist. As contributor trust decreases, some form of gatekeeping becomes more defensible. But it should always be as fast and as non-blocking as possible.

Conclusions

The evidence, while not without gaps, points in a clear and consistent direction.

Pull requests, as commonly practised -- large changes sitting in async queues for hours or days -- are an antipattern for private software teams. They were designed for a context (open source, untrusted contributors) that does not apply to most organisations. Peer-reviewed research shows they catch few bugs. Industry data shows the waiting time they impose constitutes almost all of a change’s lead time. Practitioner experience across dozens of independent voices confirms that better alternatives exist.

The alternative is not “no review.” It is better review -- built into the process rather than bolted on at the end. Test-driven development builds quality into every line. Trunk-based development keeps changes small and integration continuous. Pair and ensemble programming provide review that is immediate, contextual, and collaborative rather than delayed, decontextualised, and adversarial.

This is not about being reckless. It is about being rigorous in the right way. Deming’s insight, now more than 40 years old, still holds: you cannot inspect quality into a product. You must build it in. The teams that ship the fastest, with the fewest defects, are not the ones with the most elaborate gating processes. They are the ones that invested in the practices, skills, and trust that make gating unnecessary.

The path from where you are to where you want to be is gradual. Start by making your PRs smaller and faster. Then start asking which ones you don’t need at all. Then start pairing. Then start driving with tests. At each stage, the fear recedes a little, the flow increases, and the quality -- paradoxically, to those who equate inspection with safety -- gets better.

The question is not “How do we do pull requests better?” The question is “Why do we still need them?”

References

Peer-Reviewed Academic Research

1. Microsoft Research. “Code Reviews Do Not Find Bugs: How the Current Code Review Best Practice Slows Us Down.” IEEE, 2015. https://www.microsoft.com/en-us/research/publication/code-reviews-do-not-find-bugs-how-the-current-code-review-best-practice-slows-us-down/

2. Bacchelli, A. & Bird, C. “Expectations, Outcomes, and Challenges of Modern Code Review.” ICSE 2013. https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/ICSE202013-codereview.pdf

3. Bosu, A. et al. “Characteristics of Useful Code Reviews: An Empirical Study at Microsoft.” Microsoft Research, 2015. https://www.microsoft.com/en-us/research/publication/characteristics-of-useful-code-reviews-an-empirical-study-at-microsoft/

4. Williams, L. et al. “Strengthening the Case for Pair Programming.” IEEE Software, Vol. 17, No. 4, 2000. https://ieeexplore.ieee.org/document/854064/

5. Hannay, J.E. et al. “The Effectiveness of Pair Programming: A Meta-Analysis.” Information and Software Technology, Vol. 51, No. 7, 2009. https://www.sciencedirect.com/science/article/abs/pii/S0950584909000123

6. Arisholm, E. et al. “Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise.” IEEE Transactions on Software Engineering, Vol. 33, No. 2, 2007. https://ieeexplore.ieee.org/document/4052584/

7. Muller, M.M. & Tichy, W.F. “Two Controlled Experiments Concerning the Comparison of Pair Programming to Peer Review.” Journal of Systems and Software, Vol. 78, No. 2, 2005. https://www.sciencedirect.com/science/article/abs/pii/S0164121205000038

8. Madeyski, L. “An Empirical Study on Design Quality Improvement from Best-Practice Inspection and Pair Programming.” Springer LNCS Vol. 4034, 2006. https://link.springer.com/chapter/10.1007/11767718_27

9. di Bella, E. et al. “Pair Programming and Software Defects -- A Large, Industrial Case Study.” IEEE TSE, Vol. 39, No. 7, 2013. https://ieeexplore.ieee.org/document/6331491/

10. Nagappan, N. et al. “Realizing Quality Improvement Through Test Driven Development.” Microsoft/IBM study. https://www.researchgate.net/publication/258126622_How_Effective_is_Test_Driven_Development

11. Edmondson, A. “Psychological Safety and Learning Behavior in Work Teams.” Administrative Science Quarterly, 1999.

Industry Reports and Guides

12. Forsgren, N., Humble, J., Kim, G. Accelerate: The Science of Lean Software and DevOps. IT Revolution, 2018.

13. DORA/Google. “Capabilities: Trunk-based Development.” https://dora.dev/capabilities/trunk-based-development/

14. Google Cloud. “Accelerate State of DevOps Report 2023.” https://dora.dev/research/2023/dora-report/

15. ThoughtWorks. “Peer Review Equals Pull Request.” Technology Radar, April 2021. https://www.thoughtworks.com/radar/techniques/peer-review-equals-pull-request

16. SmartBear/Cisco. “Code Review at Cisco Systems.” 2006. https://static0.smartbear.co/support/media/resources/cc/book/code-review-cisco-case-study.pdf

Practitioner Sources

17. Deming, W.E. Out of the Crisis. MIT Press, 1982. Also: Deming Institute, “Dr. Deming’s 14 Points for Management.” https://deming.org/explore/fourteen-points/

18. Deming Institute. “Software Code Reviews from a Deming Perspective.” https://deming.org/software-code-reviews-from-a-deming-perspective/

19. de Pauw, T. “The Good and the Dysfunctional of Pull Requests.” ThinkingLabs, 2024. https://thinkinglabs.io/articles/2024/02/22/the-good-and-the-dysfunctional-of-pull-requests.html

20. de Pauw, T. “Non-Blocking, Continuous Code Reviews -- A Case Study.” ThinkingLabs, 2023. https://thinkinglabs.io/articles/2023/05/02/non-blocking-continuous-code-reviews-a-case-study.html

21. Stepanovic, D. “Async Code Reviews Are Killing Your Company’s Throughput.” 2021-2023. https://www.slideshare.net/kobac/async-code-reviews-are-killing-your-companys-throughput-248758692

22. Troy, C. “Reviewing Pull Requests.” chelseatroy.com, 2019. https://chelseatroy.com/2019/12/18/reviewing-pull-requests/

23. Morris, K. “Why Your Team Doesn’t Need to Use Pull Requests.” infrastructure-as-code.com, 2021. https://infrastructure-as-code.com/posts/pull-requests.html

24. Fowler, M. “bliki: Pull Request.” https://martinfowler.com/bliki/PullRequest.html

25. Fowler, M. “Continuous Integration.” Updated 2024. https://martinfowler.com/articles/continuousIntegration.html

26. Farley, D. “Continuous Integration and Feature Branching” (blog)

    https://www.davefarley.net/?p=247

27. / “You NEED to Stop Using Pull Requests” (YouTube video).

28. Gorman, J. “Pull Requests, Defensive Programming -- It’s All About Trust.” Codemanship, 2020. https://codemanship.wordpress.com/2020/09/12/pull-requests-defensive-programming-its-all-about-trust/

29. Vocke, H. “You Might Be Better Off Without Pull Requests.” https://hamvocke.com/blog/better-off-without-pull-requests/

30. Beck, K. (via Twitter/X): “Pull requests are an improvement on working alone but not on working together.”

31. Kerr, J. “Those Pesky Pull Request Reviews.” jessitron.com, 2021. https://jessitron.com/2021/03/27/those-pesky-pull-request-reviews/

32. Majors, C. “How Much Is Your Fear of Continuous Deployment Costing You?” charity.wtf, 2021. https://charity.wtf/2021/02/19/how-much-is-your-fear-costing-you/

33. Reinertsen, D. The Principles of Product Development Flow. Celeritas Publishing, 2009.

34. Wilsenach, R. “Ship / Show / Ask.” Published on martinfowler.com. https://martinfowler.com/articles/ship-show-ask.html

35. Hammant, P. “Trunk Based Development.”

    https://trunkbaseddevelopment.com/

36. Minimum CD. “Minimum Viable Continuous Delivery.”

    https://minimumcd.org/

37. Meek, J. “Pull Requests are an Anti-Pattern.” Substack.

38. Zuill, W. “Mob Programming: A Whole Team Approach.”

    https://mobprogramming.org/

39. Beck, K. Test-Driven Development: By Example. Addison-Wesley, 2002.

40. Humble, J. & Farley, D. Continuous Delivery. Addison-Wesley, 2010.

41. Bogard, J. “Trunk-Based Development or Pull Requests: Why Not Both?” https://www.jimmybogard.com/trunk-based-development-or-pull-requests-why-not-both/

Over the past decade, rigorous evidence — spanning Google’s Project Aristotle, the DORA research programme, academic studies, and hard-won lessons from companies like Spotify, Amazon, and GitLab — has converged on a surprisingly consistent set of themes. These studies don’t all measure the same things or use the same methods, and they don’t agree on everything. What follows is not a unified theory of team performance — it’s a curated synthesis of the strongest evidence available. Some of the conclusions are intuitive. Others are deeply counterintuitive. All of them are grounded in data, though the strength of that grounding varies, and I’ll try to be honest about where.

It’s Not About the Stars — It’s About the Dynamics

One of the most influential studies in this space is Google’s Project Aristotle, a two-year internal study of 180 teams examining hundreds of attributes. The single most critical factor in team success was not individual talent, technical skill, or seniority — it was psychological safety, defined by Harvard professor Amy Edmondson as “a shared belief held by members of a team that the team is safe for interpersonal risk taking.”

Google identified five key dynamics, in decreasing order of importance: psychological safety, dependability, structure and clarity, meaning, and impact. What did not matter in that study, with those measures was equally revealing: colocation, consensus decision-making, individual extroversion, individual performance, workload size, seniority, team size, and tenure showed no significant correlation with team effectiveness. That doesn’t mean these factors never matter — anyone who’s managed a distributed team or scaled a 30-person department knows better. But within Google’s context, they weren’t the differentiators.

This finding challenges the persistent myth of the “10x engineer.” The evidence says that who is on the team matters far less than how the team works together. Edmondson’s foundational 1999 study — one of the most cited papers in organisational psychology — established that psychological safety enables learning behaviour, which in turn drives performance improvement. The DORA research programme validated the connection: generative cultures that include psychological safety predict both software delivery performance and organisational performance.

An important nuance: psychological safety enables performance, but it doesn’t cause it on its own. Plenty of teams feel safe and still don’t ship anything useful. Safety without standards, direction, or healthy pressure can produce very comfortable mediocrity. That’s why Google’s research found it was the foundation — the other four dynamics (dependability, structure, meaning, impact) stack on top. Safety alone is not the goal. Safety in service of high expectations is.

The practical implication is profound. Hiring alone won’t get you there. You have to build the conditions for high performance, not just recruit for it.

Culture Is Not a Perk — It’s a Performance Predictor

Sociologist Ron Westrum identified three types of organisational culture: pathological (power-oriented, fear-based), bureaucratic (rule-oriented, turf-protecting), and generative (performance-oriented, mission-focused). The DORA research demonstrated that generative culture predicts software delivery performance, organisational performance, and job satisfaction.

Generative cultures are characterised by good information flow, high cooperation and trust, bridging between teams, and conscious inquiry. But here’s what makes this finding actionable: the research shows that adopting technical practices — continuous integration, test automation, trunk-based development — actively shifts culture over time. Organisations do not need to “fix culture first.” The relationship is bidirectional: culture enables technical practices, and technical practices shape culture.

A word of caution, though. This bidirectional claim is easier to write than to live. In messy organisations — the ones with blame-heavy post-mortems, territorial middle managers, and approval chains that strangle momentum — trying to introduce trunk-based development or continuous delivery can backfire. The practice gets adopted in name only, or it surfaces conflicts that the culture isn’t ready to handle. The research says the path exists; it doesn’t say the path is clean. In reality, it’s often political, uncomfortable, and slow. But the evidence that it can work, even if it doesn’t always, is worth something.

Size Matters — But Not the Way You Think

Amazon’s two-pizza teams, Hackman and Vidmar’s research, Scrum guidance, and Dunbar’s number all point in the same direction: the optimal team size is somewhere around 5 to 9 members. These sources come from different domains — organisational psychology, military anthropology, practitioner heuristics, corporate operating models — and they aren’t studying the same phenomenon in the same way. But the convergence is suggestive. The underlying mechanism is well-understood: communication channels grow quadratically — a 10-person team has three times the channels of a 6-person team. Beyond that threshold, productivity per person declines as coordination costs overwhelm individual contributions.

And when a project is already late? Fred Brooks’s 1975 observation remains validated by five decades of experience: “adding manpower to a late software project makes it later.” The ramp-up time for new members, combined with the explosion in communication overhead and the non-divisibility of many software tasks, means that the intuitive response — throw more people at the problem — is usually the wrong one.

This applies to team composition as well — but with an essential caveat. Cognitive diversity drives innovation only when combined with psychological safety and inclusive practices. Without those conditions, diverse teams may actually underperform homogeneous ones. When the conditions are right, however, the effects are striking: research published by the National Institutes of Health found that cognitively diverse teams solve complex tasks significantly faster and produce more innovative solutions. The implication is that hiring for diversity must be accompanied by investing in inclusive team dynamics — one without the other can backfire.

Your Software Architecture Is Your Org Chart (and Vice Versa)

In 1967, Melvin Conway observed that organisations design systems that mirror their communication structures. Nearly sixty years later, MIT and Harvard Business School researchers found “strong evidence to support the mirroring hypothesis” — products built by loosely-coupled organisations are significantly more modular than those from tightly-coupled ones.

This has moved from observation to strategic tool. Martin Fowler endorses the “Inverse Conway Manoeuvre” — deliberately structuring teams to produce the desired architecture. Matthew Skelton and Manuel Pais built the entire Team Topologies framework on this principle, defining four team types: stream-aligned (delivering direct customer value), platform (reducing cognitive load via self-service), enabling (coaching other teams temporarily), and complicated-subsystem (handling specialist technical components).

The key insight from Team Topologies is that the primary benefit of a platform team is not efficiency — it’s reducing the cognitive load on stream-aligned teams. When cognitive load is managed, teams can focus on their core mission. When it isn’t, every team becomes a bottleneck for every other team.

Measure What Matters — And Only What Matters

The DORA research programme originally identified four key metrics that predict software delivery performance: deployment frequency, lead time for changes, change failure rate, and time to restore service. A fifth metric — reliability — was added in subsequent years, and the 2024 report further refined the framework by replacing “mean time to recovery” with failed deployment recovery time (moving it from stability to throughput) and introducing rework rate as a new stability measure.

From 2018 through 2024, DORA grouped teams into four performance clusters — elite, high, medium, and low — derived each year via cluster analysis of survey respondents. The table below, based on the original Accelerate benchmarks, illustrates the scale of the gap between top and bottom performers:

ClusterDeploy FrequencyLead TimeFailure RateRecovery TimeEliteOn demand< 1 hour~5%< 1 hourHighDaily to weekly1 day - 1 week~10%< 1 dayMediumWeekly to monthly1 week - 1 month~15%1 day - 1 weekLowMonthly+1 month+~64%1 week+

In 2025, DORA retired these tiers entirely and replaced them with seven team archetypes — such as “Harmonious High Achiever,” “Stable and Methodical,” and “Legacy Bottleneck” — that blend delivery metrics with human factors like burnout and friction. The shift reflects a recognition that a single linear ranking oversimplifies how teams actually perform: a team can have high throughput but crushing burnout, or modest deployment frequency but exceptional stability. For practitioners, the implication is significant: “high-performing” is no longer a single target to aim for. It means understanding which archetype your team resembles and addressing its specific weaknesses, rather than chasing a universal ideal. The field is moving from “are we elite?” toward “what kind of team are we, and what’s holding us back?”

Regardless of how the clusters are drawn, one finding has remained consistent across every year of the research: throughput and stability are not trade-offs — the highest-performing teams excel at both. This demolishes the common excuse that moving fast requires sacrificing quality. If you’re deploying monthly, aim for weekly first — not “on demand.”

But beware the wrong metrics. McKinsey’s 2023 attempt to measure individual developer productivity generated fierce backlash from the engineering community, with critics including Gergely Orosz and Kent Beck arguing the framework would “do far more harm than good.” Among engineering leaders, the prevailing view is that individual-level productivity metrics are counterproductive and damage trust. Measure teams and systems, not individuals. The SPACE framework — Satisfaction, Performance, Activity, Communication, and Efficiency — formalises the principle that developer productivity cannot be reduced to a single dimension or metric. At least three of the five dimensions should be measured simultaneously to avoid misleading conclusions.

The Practices That Separate the Best From the Rest

Technical practices are not just implementation details — they are culture shapers. The Accelerate research found that teams practising trunk-based development deploy over two hundred times more frequently with over a hundred times faster lead times than the lowest-performing teams. Teams that work off trunk or branches lasting less than a day have significantly higher performance.

A necessary caveat: most of this evidence is correlational, not causal. High-performing teams practice trunk-based development, but high-performing teams also tend to be the ones capable of adopting these practices well. The arrow of causation isn’t always clear. Treat these practices as strongly associated with high performance rather than as guaranteed recipes for it. And note that these practices done badly can actively hurt: trunk-based development without solid CI and feature flags leads to broken mainlines; test automation without discipline produces brittle suites that slow everything down and erode trust in the tests themselves.

Code review velocity matters too. Google’s code review practices demand a maximum one-business-day response time, and the overwhelming majority of their engineers report satisfaction with the process. The primary purpose is not gatekeeping — it’s “making sure that the overall code health of Google’s code base is improving over time.” Speed and quality in code review are not trade-offs. Fast turnaround prevents context-switching costs and unblocks developers, while the review itself maintains code quality and spreads knowledge.

Test automation is another proven driver. When Google Web Server hit a crisis in 2005 — 80% of pushes causing user-impacting bugs — mandatory test automation and the “Test Certified” programme resolved the crisis within two years. Mike Cohn’s test pyramid (many unit tests, fewer integration tests, few end-to-end tests) remains the most validated testing strategy.

Pair programming, by contrast, shows mixed results. A meta-analysis found a small positive effect on quality and a medium positive effect on duration, but a medium negative effect on effort. Quality benefits are strongest for complex tasks. The evidence supports pair programming as a tool to deploy selectively — for complex tasks, knowledge transfer, and onboarding — not as a mandatory practice.

Autonomy With Alignment — The Balancing Act

High-performing teams need both freedom and direction. OKRs (Objectives and Key Results) provide the mechanism: at Google, the majority of OKRs are developed bottom-up and then aligned to company goals. Fully top-down OKRs reduce autonomy; fully bottom-up OKRs risk misalignment. This balance enables what the DORA research calls “loosely coupled architecture with tightly aligned goals.”

Spotify’s organisational model — squads, tribes, chapters, guilds — offers a cautionary tale about the limits of structure. Spotify themselves have publicly acknowledged the model never fully worked as described. Tribes became siloed, cross-tribe coordination proved difficult, guilds lost effectiveness at scale, and high autonomy without collaboration processes led to duplicated effort. Companies that adopted the structure without the underlying culture of trust and psychological safety consistently failed.

The lesson: organisational structure is necessary but insufficient. It works only when underlaid by generative culture. You can’t copy another company’s org chart and expect their results.

Developer Experience Is a Strategic Lever

The DevEx framework, published in ACM Queue in 2023, distils developer experience to three core dimensions: feedback loops (speed and quality of responses to developer actions), cognitive load (mental processing required for tasks), and flow state (energised focus, full involvement, and enjoyment).

All three are actionable. Feedback loops can be measured through build times, CI/CD pipeline duration, and code review turnaround. Cognitive load can be reduced through better documentation, simpler architecture, and fewer context switches. Flow state can be enabled by reducing interruptions and providing blocks of uninterrupted time.

Asynchronous communication plays a role here. GitLab, a fully remote company with 2,000+ employees, operates on an “async-first” basis. Async communication minimises distractions and enables deeper focus, but success requires deliberate investment in documentation and clear communication guidelines. The trade-off is real: async can slow decision-making and reduce personal connection if not managed carefully.

Invest in Beginnings and Belonging

According to widely-cited research, organisations with strong onboarding processes see dramatically higher new hire retention and productivity — though these figures originate from a single industry study and should be treated as directional rather than precise. More robustly sourced: Google found that new hires paired with buddies reached full efficiency significantly faster, and one company reduced time-to-productivity from six weeks to ten days through structured onboarding with automation.

The most effective onboarding programmes share common elements: hardware and access configured before day one, a buddy or mentor pairing, a 30/60/90-day milestone framework, a first meaningful commit within the first week, and architecture documentation that explains why, not just how.

Mentorship extends beyond onboarding. Deloitte found that millennials with mentors are roughly twice as likely to stay at their organisation long-term, and employees involved in mentoring programmes have markedly higher retention rates. The evidence broadly supports mentorship as beneficial, though specific, evidence-based guidance for implementing it in software engineering contexts remains a real gap. Most of the research is cross-industry, and what works for management consulting or sales may not transfer directly to a codebase with steep domain complexity. Teams should treat mentorship as a high-probability bet worth making, while measuring retention and ramp-up outcomes locally rather than assuming the general statistics apply.

The Anti-Patterns That Destroy Performance

Knowing what to avoid is as important as knowing what to do.

Burnout is not an individual weakness — it’s an organisational design problem. The Accelerate research identifies six organisational risk factors: work overload, lack of control, insufficient rewards, breakdown of community, absence of fairness, and value conflicts. The vast majority of developers report experiencing work-related burnout. The 2024 DORA report found a specific, actionable intervention: teams with stable priorities face significantly less of it.

Measuring developers by lines of code, commit count, or individual output metrics creates perverse incentives and damages team culture. Organisations with generative culture — which avoids such metrics — consistently outperform those that don’t. The SPACE framework was explicitly designed as an alternative to these reductive metrics.

Platform engineering can backfire. The 2024 DORA report found measurable decreases in both throughput and change stability when teams were required to exclusively use internal platforms. The benefit comes from voluntary adoption of well-designed platforms, not mandated use.

And AI? The early data is more nuanced than the hype suggests. The 2024 DORA report found that as AI adoption increases, delivery stability drops measurably — even as developers feel more productive. One plausible interpretation — though not yet proven — is that AI-assisted coding tends to produce larger changesets, and larger changesets introduce more risk. Without AI-specific code review practices and batch size discipline, teams ship more code but break more things. This is not an argument against AI adoption — it’s an argument for treating it as a practice that requires the same measurement rigour as any other. Monitor your change failure rate and batch sizes before and after AI rollout. If stability drops, slow down and adjust.

The Bottom Line

Before listing what the research points to, it’s worth naming what it largely leaves out: the external forces that often dominate outcomes more than any internal practice. Funding models, leadership churn, regulatory pressure, market dynamics, reorgs — these shape what teams can actually do far more than whether they’ve adopted trunk-based development. The research tends to study teams in relatively stable contexts. If your organisation is in the middle of a merger, a layoff, or a pivot, the advice here still applies in principle, but the path to applying it is much harder and much more political than the research alone suggests.

With that caveat, there are clear patterns across these studies, even if the details vary. High-performing software teams are not built by assembling the most talented individuals. They are built by creating the conditions in which ordinary professionals can do extraordinary work together.

Those conditions are:

1. Psychological safety as the foundation — people must feel safe to take risks, ask questions, and admit mistakes

2. Generative culture — mission-focused, high-trust, with good information flow

3. Small, cognitively diverse teams — 5 to 9 members with managed cognitive load

4. Technical excellence — CI/CD, trunk-based development, test automation, fast code reviews

5. Autonomy with alignment — bottom-up goals connected to organisational mission

6. Great developer experience — fast feedback loops, low cognitive load, protected flow state

7. Investment in people — structured onboarding, mentorship, inclusive practices

8. The right metrics — team-level outcomes, not individual activity

The relationship between these elements is not linear. Technical practices shape culture. Culture enables technical practices. Structure constrains architecture. Architecture constrains structure. The organisations that succeed are the ones that treat all of these as a system, not a checklist.

If I had to distil it even further, the research points to what I think of as T*D — three practices that, taken together, capture the essence of high-performing teams. Trunk-based development: integrate continuously, keep branches short-lived, and ship in small increments. Test-driven development: build quality in from the start rather than inspecting it in at the end. And team-focused development — what some call social programming — pairing, mobbing, and ensemble work that spreads knowledge, reduces bus-factor risk, and turns code review from a bottleneck into a conversation. Each of these reinforces the others. Trunk-based development demands good tests. Good tests demand shared understanding. Shared understanding comes from working together. T*D is where technical excellence and team culture meet.

The good news? You don’t have to fix everything at once. Pick one practice. Implement it well. Measure the result. The evidence says that even small changes — adopting trunk-based development, speeding up code reviews, running genuine retrospectives with follow-through — create virtuous cycles that build momentum over time.

The evidence is strong, but it’s suggestive rather than prescriptive. Context matters enormously. The hard part was never knowing the practices — it’s navigating the trade-offs in your specific environment, with your specific constraints, and your specific people. The research won’t make those decisions for you. But it can tell you which directions have worked for others, and which ones haven’t. That’s worth paying attention to.

References

1. AWS Executive Insights. “Amazon’s Two Pizza Teams.” aws.amazon.com.

2. Martin Fowler. “Two Pizza Team.” martinfowler.com.

3. Mountain Goat Software. “The Just-Right Size for Agile Teams.” mountaingoatsoftware.com.

4. Robin Dunbar. “Dunbar’s Number.” Referenced via psychsafety.com and Wikipedia.

5. Fred Brooks. “The Mythical Man-Month: Essays on Software Engineering.” Addison-Wesley, 1975.

6. Alan MacCormack, John Rusnak, Carliss Baldwin. “Exploring the Duality between Product and Organizational Architectures.” Harvard Business School.

7. Nagappan, Murphy, Basili. “The Influence of Organizational Structure on Software Quality.” University of Maryland.

8. “Conway’s Law Revisited: The Evidence for a Task-Based Perspective.” IEEE Software.

9. Matthew Skelton, Manuel Pais. “Team Topologies: Organizing Business and Technology Teams for Fast Flow.” IT Revolution, 2019.

10. Nicole Forsgren, Jez Humble, Gene Kim. “Accelerate: The Science of Lean Software and DevOps.” IT Revolution, 2018.

11. Amy Edmondson. “Psychological Safety and Learning Behavior in Work Teams.” Administrative Science Quarterly, 44(2), 1999.

12. Google re:Work. “Guide: Understand team effectiveness.” rework.withgoogle.com.

13. DORA. “Capabilities: Generative organizational culture.” dora.dev.

14. Ron Westrum. “A typology of organisational cultures.” BMJ Quality & Safety, 2004.

15. IT Revolution. “Westrum’s Organizational Model in Technology Organizations.” itrevolution.com.

16. Boston Consulting Group. “How Diverse Leadership Teams Boost Innovation.”

17. PMC/NIH. “When and how is team cognitive diversity beneficial?” pmc.ncbi.nlm.nih.gov.

18. ACM Digital Library. “Diversity and Teamwork in Student Software Teams.” dl.acm.org.

19. DORA. “DORA’s software delivery performance metrics.” dora.dev.

20. DORA. “Accelerate State of DevOps Report 2024.” dora.dev.

21. Google Cloud Blog. “Use Four Keys metrics to measure your DevOps performance.” cloud.google.com.

22. Forsgren, Storey, Maddila, Zimmermann, Houck, Butler. “The SPACE of Developer Productivity.” ACM Queue, Vol 19(1), 2021.

23. Gergely Orosz. “Measuring developer productivity? A response to McKinsey.” newsletter.pragmaticengineer.com.

24. McKinsey. “Developer Velocity: How software excellence fuels business performance.” mckinsey.com.

25. InfoQ. “Trunk Based Development as a Cornerstone for Continuous Delivery.” infoq.com.

26. Google Engineering Practices. “The Standard of Code Review.” google.github.io.

27. Google. “Code Review - Software Engineering at Google.” abseil.io.

28. Hannay, Dyba, Arisholm. “The effectiveness of pair programming: A meta-analysis.” Information and Software Technology, 2009.

29. Mike Cohn. “Succeeding with Agile: Software Development Using Scrum.” Addison-Wesley, 2009.

30. Google re:Work. “Guide: Set goals with OKRs.” rework.withgoogle.com.

31. John Doerr. “Measure What Matters.” Portfolio/Penguin, 2018.

32. Henrik Kniberg, Anders Ivarsson. “Scaling Agile @ Spotify.” Crisp, 2012.

33. Noda, Storey, Forsgren, Greiler. “DevEx: What Actually Drives Productivity.” ACM Queue, 2023.

34. GitLab Handbook. “How to embrace asynchronous communication for remote work.” handbook.gitlab.com.

35. Cortex. “Developer Onboarding Guide.” cortex.io.

36. Deloitte. Mentorship and retention research. Referenced via guider-ai.com.

37. OSTI. “An Exploration of the Mentorship Needs of Research Software Engineers.” osti.gov.

38. ScienceDirect. “Burnout in software engineering: A systematic mapping study.” 2022.

39. Ellahi, Rehman, Javed, Sultan, Rehman. “Impact of Servant Leadership on Project Success.” SAGE Open, 2022.

40. Spotify Engineering. “Squad Health Check model.” engineering.atspotify.com, 2014.

41. Patrick Lencioni. “The Five Dysfunctions of a Team.” Jossey-Bass, 2002.

42. ADR GitHub. “Architectural Decision Records.” adr.github.io.

43. Spotify Engineering. “When Should I Write an Architecture Decision Record.” 2020.

44. PMC. “Perceived diversity in software engineering: a systematic literature review.” 2021.

45. Martin Fowler. “Team Topologies.” martinfowler.com.

46. Inc. “Google Spent 2 Years Studying 180 Teams.” inc.com.

47. LeaderFactor. “Project Aristotle Psychological Safety.” leaderfactor.com.

48. LaunchDarkly. “Elite Performance with Trunk-based Development.” launchdarkly.com.

49. Dr. Michaela Greiler. “Code Reviews at Google are lightweight and fast.” michaelagreiler.com.

50. LeadDev. “What McKinsey got wrong about developer productivity.” leaddev.com.

51. GetDX. “Highlights from the 2024 DORA State of DevOps Report.” getdx.com.

52. ScienceDirect. “Burnout in software engineering: A systematic mapping study.” 2022.

53. DORA. “State of AI-assisted Software Development 2025.” dora.dev.

54. Kent Beck. “Measuring developer productivity? A response to McKinsey.” Substack, 2023.

He is not fine. He is stuck. And no dashboard would ever show it.

[If you have lived a version of this moment — and if you have managed engineers for any length of time, you almost certainly have — then you already know what this article is about.]

That is the core problem this article is about. Not that measurement is bad — measurement is essential — but that most of what we measure in software engineering tells us almost nothing about the thing we actually need to understand: whether our people are growing, struggling, or coasting. The question is not whether to assess your developers. It is whether your assessment tools are capable of seeing what matters.

The measurement problem

Most managers, when asked how they evaluate their developers, will point to some combination of activity metrics: tickets closed, story points delivered, lines of code written, pull requests merged. These feel scientific. They feel objective. And as tools for evaluating individuals, they are actively counterproductive.

There is a name for this. Goodhart’s Law states that when a measure becomes a target, it ceases to be a good measure. In software engineering, this plays out predictably: if lines of code become a performance indicator, developers produce verbose, unnecessarily complex code. If story points become the yardstick, estimates get inflated. If pull requests merged is the metric, people split work into trivially small changes to pump the numbers. When McKinsey published a paper in 2023 claiming they could measure individual developer productivity through activity metrics, Dan North called the thinking “absurdly naive.” Kent Beck warned that such measurements create perverse incentives — managers pressure for better scores, engineers game the system, and the actual work suffers.

But the answer is not to stop measuring. It is to measure the right things, at the right level, for the right purpose.

Metrics like deployment frequency, lead time for changes, change failure rate, and mean time to recovery — the DORA metrics, validated by Forsgren, Humble, and Kim across 23,000 data points over four years of research — are genuinely useful. They tell you about the health of the system: how quickly value flows from idea to production, how often things break, and how fast the team recovers. What they do not tell you is which individuals are thriving and which are struggling. That requires a different kind of assessment entirely — one that is richer, more contextual, and necessarily more human.

The distinction matters enormously. Metrics for understanding how a system is performing are productive. Metrics for evaluating individual people are destructive. Everything in this article sits on that line.

Watch people work

The richest source of signal about a developer is watching them work. Not in a surveillance sense, but through practices like pair programming and mob programming. When you are pairing with someone, or watching them pair with others, you learn things within a few sessions that no dashboard could ever reveal. You see how they think through problems, how they break down complexity, whether they write tests first or bolt them on as an afterthought, how they respond when their assumptions are wrong, and how they communicate with the person next to them. Academic studies on pair programming confirm it reveals individual skill levels, communication ability, and problem-solving approaches in ways no other assessment method matches.

There is a catch, and it is worth naming upfront. Harvard researcher Ethan Bernstein demonstrated through a rigorous field experiment that being observed can counterintuitively make people worse at their jobs — not because they are lazy and get caught, but because they hide the creative, experimental behaviour that produces the best work. I will return to this tension later, because it is real and it does not have a tidy resolution. But the alternative — assessing people from a distance through thin numerical proxies — is not just less effective. It is actively misleading.

Pay attention to how quickly code gets from someone’s machine to production. If a developer is sitting on branches for days, that tells you something. If their work integrates smoothly and frequently, that tells you something else entirely. The ability to work in small, safe increments is one of the most reliable markers of engineering maturity. The Accelerate research confirms this: elite teams deploy code multiple times per day, while low performers deploy once per month or less. Working in small batches reduces cycle times, accelerates feedback, reduces risk, and improves efficiency. And here is the finding that surprises many managers: speed and stability are not trade-offs. Teams that deploy more frequently also have lower failure rates and faster recovery times.

One caveat matters here. A developer sitting on branches for days might not be signalling personal weakness. The CI/CD pipeline might be broken. Code review might be bottlenecked. The team might lack trunk-based development practices. Before judging the individual, check the system. This is a principle that runs through the entire article: whenever you see individual behaviour that looks like a problem, ask whether the environment is causing it before concluding that the person is.

Watch how people behave when something breaks. Good developers do not panic and do not blame. They have a systematic approach to diagnosis. They narrow things down. They ask good questions. And crucially, after they fix the problem, they ask “how do we make sure this class of problem cannot happen again?” Google’s Project Aristotle — a two-year study of over 180 teams — found that psychological safety was the single strongest predictor of team performance, accounting for 43 percent of the variance. Teams where people blame and panic are teams where psychological safety is low. Teams where people diagnose calmly and think about prevention are teams where it is high.

Then there is the question of learning. Good developers are visibly curious. They read, they experiment, they ask “why” a lot. But here is the nuance: you are not looking for people who chase every shiny new framework. You are looking for people who deepen their understanding of fundamentals. Someone who properly understands testing, design, and feedback loops will pick up any new tool in a week. Someone who only knows tools but not principles will struggle every time the landscape shifts. Daniel Pink’s research on motivation calls this mastery — the desire to continually improve at something that matters. It is one of three core intrinsic motivators for knowledge work, alongside autonomy and purpose. The developers who chase depth are driven by mastery. The ones who chase breadth without depth are often driven by anxiety.

That said, this is not a binary. In a fast-moving field, some exploration of new tools is adaptive and necessary. The signal is not whether someone looks at new things, but whether they can evaluate them critically against fundamentals. Curiosity about new tools combined with deep understanding of principles is the strongest combination.

Practical actions for managers

Let us get concrete about what a manager can actually do day to day to understand their team, not in a way that punishes or controls, but in a way that genuinely helps.

Sit with your teams regularly. Not in a “let me observe you” way, but genuinely participate. Join a mob session. Pair with someone on a real task. Even if you are not coding yourself, being in the room while work happens gives you an enormous amount of information. You will see who drives the design conversation, who asks clarifying questions, who just waits to be told what to do, and who challenges assumptions constructively. Do this often enough and you will build a mental model of every person on the team that no spreadsheet could ever give you.

Pay attention to what happens after someone’s code merges. Does it cause problems downstream? Do other people frequently have to rework it? Or does it land cleanly and become something the team builds on with confidence? You do not need a fancy tool for this. Just ask the team during retrospectives: “What slowed us down this week?” If the same person’s work keeps coming up as a source of friction, that is a signal. If someone’s work is invisible because it just works, that is also a signal, and you should notice it.

Give people a small, ambiguous problem and see what they do with it. Not as a test, but as real work. Something like “we have had three customer complaints about this area of the product, can you investigate and come back with a recommendation?” A strong developer will dig in, ask smart questions, maybe prototype something, and come back with options. Someone less experienced, or someone who has not been given the space to develop this skill, will either freeze, ask you to define every detail upfront, or jump straight to coding without understanding the problem. This tells you far more than any technical interview ever could, because it is real. Will Larson’s research on staff engineering identifies exactly this — the capacity to self-direct through uncertainty rather than waiting for someone to remove it — as the defining characteristic that separates mid-level from senior engineers, and senior from staff.

Look at how people handle knowledge gaps. Give someone a task that is slightly outside their comfort zone and watch what happens. Do they go and learn what they need? Do they ask for help at the right moment, not too early and not after they have wasted three days? Do they share what they learned afterwards? The ability to self-direct learning and then bring it back to the team is one of the strongest indicators of someone who will grow into the role versus someone who has plateaued.

Have honest one-to-ones where you ask open questions and then actually listen. Not “how are your tickets going” but things like “what is the hardest technical decision you have made recently and why?” or “what part of our codebase worries you most?” or “if you could change one thing about how we work, what would it be?” The quality of the answers tells you a lot. Someone who engages thoughtfully with these questions is someone who cares about the craft. Someone who consistently has nothing to say may not be thinking deeply about the work — or they may not yet trust that it is safe to say what they think. Some people are internal processors. Some come from cultures where questioning how the team works is not done casually. The manager’s job is to figure out which it is, not to assume the worst. If someone goes quiet in one-to-ones, the first question should be “have I made this space safe enough?” not “does this person care?”

And get feedback from peers. Not formal 360 reviews with scores and ratings used for appraisal — research shows those tend to produce manipulated data, with gaming documented at companies including GE, IBM, and Amazon. But do not dismiss structured peer feedback entirely. When 360-degree feedback is used for development rather than evaluation, and when responses are anonymous, research shows it genuinely increases communication, productivity, and team effectiveness. The best approach combines informal peer conversations — “What is it like working with Sarah?” — with structured but anonymous developmental feedback that lets people say things they would not say to someone’s face. Some of the most important signals, especially upward feedback about a manager’s own behaviour, only surface when anonymity is guaranteed.

Being honest about scale

Everything I have described so far sounds reasonable for a manager with five or six direct reports. But what if you have twelve? What if you manage two teams across different time zones? The honest answer is that you cannot pair with every engineer every week, attend every mob session, and have deep one-to-ones with everyone on a fortnightly cycle. If this approach only works at small scale, it is not a complete approach.

There are three things that make it practical at scale.

First, you do not need to be the only observer. Tech leads, senior engineers, and other experienced team members are already watching the work. They see how people pair, how they handle ambiguity, how they respond to feedback. Your job is not to personally observe everything but to build a network of people whose judgement you trust and to triangulate their perspectives with your own. Ask your tech lead: “How is Maria doing on the migration? What have you noticed?” This is not delegation of responsibility. It is distributed observation, and it produces a richer, less biased picture than a single manager’s viewpoint ever could.

Second, you can rotate your depth of focus. Rather than shallowly observing everyone all the time, spend a few weeks deeply engaged with one part of the team — joining their sessions, reading their pull requests, having longer conversations — then rotate. Over a quarter, you will have built a detailed picture of everyone. This is not ideal, but it is honest, and it is better than the alternative of seeing no one deeply.

Third, mob programming is more efficient for observation than pairing, precisely because you see the whole team’s dynamics at once. A single mob session reveals who drives, who supports, who withdraws, who challenges, and who defers. You do not need to attend many sessions to learn a lot. If you are time-constrained, mobs give you the highest signal per hour invested.

The honest tension: observation is still observation

There is a tension here that deserves honesty rather than a neat resolution. If you are joining a session partly to learn about the work and partly to assess the people doing it, then dressing it up as pure curiosity is a form of dishonesty. People are not stupid. They can feel when they are being evaluated, regardless of what words you use. And if they later discover that your “just joining in” sessions fed into a performance conversation, the trust damage is significant.

This tension is not just intuition. It has academic backing. Bernstein’s “Transparency Paradox” research, which I mentioned earlier, demonstrated this through a field experiment at a large manufacturing plant. Workers who knew they were being watched concealed innovative approaches, suppressed experimentation, and avoided productive deviation from standard practice. Bernstein found that even a modest increase in group-level privacy sustainably and significantly improved performance. The implication is uncomfortable: the very act of a manager watching can make people worse at their jobs, because they hide the creative, experimental behaviour that produces the best work.

So the better approach is transparency. Not “I am here to check on you,” but something closer to the truth: “Part of my job is understanding how the team works, what is going well, and where people might need support. I cannot do that if I am never near the work. So I am going to join sessions from time to time. Not to catch anyone out, but because I need to see reality if I am going to be any use to you.” That is honest. It acknowledges the evaluative dimension without making it adversarial.

But here is the deeper point, and Bernstein’s research supports it. If the only time a manager is near the work is when they are assessing people, then of course it feels like surveillance, no matter how it is framed. The real fix is not about how you explain your presence. It is about whether your presence is normal or exceptional. If a manager is routinely involved in technical discussions, regularly joins sessions, frequently asks about design decisions, and has ongoing conversations about the codebase, then there is no single moment that feels like “the inspection.” It is just how things work. The observation happens as a side effect of genuine involvement, not as a discrete activity with a hidden agenda.

The managers who struggle most with this are the ones who are distant from the work ninety percent of the time and then suddenly show up. In that context, no amount of friendly framing will stop people from feeling watched. The discomfort is not caused by the words. It is caused by the pattern.

There is also something worth saying about the broader culture. In teams where feedback is frequent and open, where people regularly talk about what is going well and what is not, evaluation loses its menacing quality. People feel micromanaged when assessment is something that happens to them in secret and then gets revealed in a formal review. If instead the manager is consistently saying “I noticed this went well” and “I think this area needs work” as part of normal conversation, then the evaluative aspect of being present becomes something people are used to rather than something they dread.

None of this fully resolves the tension. A manager who is present will, by definition, be forming judgements. People who are being observed will, by definition, behave somewhat differently. You cannot eliminate that dynamic entirely. But you can make it honest, make it routine, and make it part of a relationship where people trust that the judgements being formed are fair and in their interest. That is the best you can do, and it is a lot better than the alternative, which is forming judgements from a distance based on terrible data and then surprising people with them twice a year.

The biases you carry into the room

If observation is your primary assessment tool — and I am arguing it should be — then you need to be honest about the fact that you are not an objective instrument. Decades of research on performance evaluation show that managers are systematically biased in ways they are rarely aware of.

Recency bias means you overweight what happened in the last few weeks and forget the previous months. SHRM research shows that recency and central tendency errors affect nearly 40 percent of annual appraisals, and they do not disappear just because you are observing in person rather than reading a spreadsheet. That developer who had a rough week when you happened to join the mob session? You will remember that disproportionately.

Similarity bias means you unconsciously favour people who remind you of yourself — same background, same communication style, same approach to problem-solving. The developer who thinks the way you do will seem “stronger” than the one who thinks differently but equally well.

The halo and horns effects mean that one strong or weak impression colours everything else. If someone impressed you with an excellent design decision, you will be more generous when evaluating their testing practices, even if those are mediocre. If someone fumbled an incident response, you may underrate their day-to-day coding, even if it is excellent.

And then there is proximity bias, which matters more than ever. Managers form more favourable impressions of people they see frequently. In hybrid or remote teams, this creates systematic unfairness: research shows employees may receive better review outcomes simply because they work in the same office as their manager. The person you pair with every week will feel more “known” — and therefore rated more positively — than the remote team member you interact with only in stand-ups.

One of the largest studies on feedback found that more than half of the variance in performance ratings had more to do with the quirks of the person giving the rating than the person being rated. That means the biggest variable in your evaluation is you, not them.

What do you do about this? You cannot eliminate bias, but you can discipline your observation. Keep running notes. Not a surveillance dossier, but a simple habit: after a pairing session or a notable interaction, write down what you actually saw, not what you felt about it. Over time, patterns emerge from evidence rather than from impressions. Seek disconfirming evidence actively — if you think someone is weak, look specifically for moments where they are strong, and vice versa. Calibrate with peers: ask other managers or tech leads who work with the same people whether they see what you see. And be especially deliberate about equalising your attention across the team. If you are pairing more often with some people than others, you are building a biased dataset whether you mean to or not.

When the team is not in the room

Everything I have described so far assumes you can physically or synchronously join a session with your team. But many teams are distributed. Some are fully remote. Some span time zones. If observation-based assessment only works when you can sit next to someone, then it is not a complete approach.

The principles remain the same, but the channels change. In distributed teams, the work leaves more written traces, and those traces become your primary observation material.

Code review is the most revealing. Not reviewing code yourself as a gatekeeper, but reading how people engage with each other’s code. The difference between a thoughtful reviewer and a careless one is immediately visible. Compare “this is wrong, fix it” with “I think this approach might cause issues under concurrency — have you considered using a lock here? Happy to pair on it if useful.” The first tells you someone is going through the motions. The second tells you someone understands the system, communicates with care, and is willing to invest time in a colleague’s growth. How people respond to critique is equally telling: do they engage with the substance, or do they get defensive? Do they ask follow-up questions, or do they silently apply the change without understanding why? The quality of someone’s code review comments tells you as much about their engineering judgement as watching them code would.

Look at how people communicate in writing more broadly. In asynchronous teams, the ability to write a clear problem statement, a well-structured RFC, or a concise incident summary is itself a form of engineering skill. People who can articulate their thinking in writing are usually people who think clearly, and that matters regardless of whether you are ever in the same room.

Pay attention to how distributed team members handle the particular challenges of remote work. Do they proactively communicate blockers or sit silently for days? Do they make themselves available for synchronous collaboration when it matters, or are they always unavailable? Do they contribute to team discussions, or do they disappear between assigned tasks?

None of this is as rich as pairing with someone in real time. But it is far better than falling back on ticket counts and activity dashboards, which is what most managers of remote teams end up doing by default.

Promotions, recognition, and the evidence problem

This is where it gets uncomfortable, because most organisations pretend they have a system for promotions when really they have a ritual. Someone fills in a form, a manager writes a justification, a calibration meeting happens where people who have never seen the work argue about ratings, and then a decision gets made that is mostly political. Everyone involved knows it is theatre, but nobody says it out loud.

The data you need is not data in the traditional sense. It is evidence. And the best evidence comes from the practices already described, but you need to be deliberate about collecting it. Not in a creepy dossier way, but as a habit of noticing and writing things down. When someone handles an incident well, make a note. When someone’s pairing session lifts the whole team’s understanding, write it down. When someone repeatedly delivers work that needs reworking, note that too. Over time you build a picture that is grounded in real events, not vibes and not metrics.

For promotions specifically, the question should not be “has this person earned a reward?” It should be “is this person already operating at the next level?” That distinction matters enormously. If someone is already doing the work of a senior engineer, meaning they are influencing design decisions, mentoring others, taking ownership of ambiguous problems, thinking about the system rather than just their task, then promoting them is just recognising reality. If you are promoting someone because they have been around long enough or because they will leave otherwise, you are creating problems.

The evidence for this is observable. You can point to specific moments. “In the last six months, you led the redesign of the payment service. You brought three junior developers along with you through pairing. You identified the performance issue before it hit production. You pushed back on the product team when the requirements did not make sense and proposed a better alternative.” That is a promotion case built on things that actually happened, not on a self-assessment form where someone writes “I demonstrated leadership” with no context.

But there are two fairness problems here that deserve honesty.

The first is access. The “already operating at the next level” criterion only works if everyone has equal access to the opportunities that let them demonstrate next-level work. In practice, they often do not. People from underrepresented groups, quieter team members, people in less visible parts of the codebase, and remote workers may all have fewer chances to lead a high-profile redesign or push back on product in a visible way. Research on equitable promotion policies confirms that marginalised groups are promoted at a slower rate, partly because the opportunities to demonstrate next-level capability are unequally distributed.

This means managers have an active responsibility, not just to observe who steps up, but to deliberately distribute stretch opportunities. If you only promote people who naturally volunteer for visible work, you are filtering for confidence and political skill, not engineering ability. Make sure the quiet person on the team gets the chance to lead something. Make sure the remote team member gets the same ambiguous problems as the person sitting next to you.

The second is compensation. “Already operating at the next level” means, in practice, doing a harder job for months while being paid for the easier one. Most companies expect six to twelve months of sustained performance at the next level before promoting. That is six to twelve months of uncompensated labour at a higher level of responsibility. This might sound reasonable in the abstract, but it disproportionately affects people who cannot afford to “invest” unpaid effort — and it is the single most common criticism of this promotion model among the engineers who live inside it.

I do not have a clean solution for this. But I think the manager’s obligation is clear: make the gap between “doing the work” and “getting the title” as short as organisationally possible. If someone has been operating at the next level for two full review cycles and you still have not promoted them, that is a management failure, not a demonstration of rigour. And be transparent about the timeline. If someone is doing senior-level work, tell them: “I see it, I am building the case, and here is when I expect it to happen.” Silence on this topic is how you lose your best people.

Feedback that actually changes behaviour

For feedback conversations, the same principle applies. Be specific and be timely. “Your code in the checkout service last week had no tests and broke the build twice” is useful feedback. “You need to improve your quality” is not. The first gives someone something to act on. The second just makes them feel bad.

Here is something most managers get wrong: feedback should not be saved up for a quarterly review. If you see something that needs addressing, address it within days, not months. If you see something excellent, say so immediately. The idea that feedback is a formal event is one of the most damaging conventions in management. It means people spend months not knowing where they stand, which breeds anxiety and kills trust.

The research on this is overwhelming. Gallup found that employees who receive feedback weekly are 2.7 times more likely to be engaged at work. More strikingly, employees are 3.6 times more likely to be motivated to do outstanding work when their manager provides daily feedback compared to annual feedback. Companies with strong feedback cultures see 14.9 percent lower turnover. When Adobe shifted from annual reviews to continuous check-ins, they saw a 30 percent drop in voluntary turnover within a single year.

If you are giving feedback regularly, the formal review becomes a summary of things both of you already know, which is exactly what it should be. Nothing in a performance review should ever be a surprise. If it is, you have failed as a manager, not because the assessment is wrong, but because you waited too long to share it.

On recognition, be careful about making it purely individual. Software is a team activity. If you only recognise individual heroes, you incentivise hero behaviour, and that is corrosive. Hero culture leads to knowledge silos, bus factor risks, and burnout among the very people being celebrated. Perhaps most damningly, research suggests that hero culture is itself a sign of low psychological safety — when only heroes are recognised, others stop taking initiative because the implicit message is that only extraordinary individual acts matter.

Google’s Project Aristotle found that the number one predictor of team performance was not individual talent but psychological safety. Teams with high psychological safety showed 19 percent higher productivity, 31 percent more innovation, and 27 percent lower turnover. Individual heroics were not on the list.

Recognise teams. Recognise collaborative moments. “The way you and Tom worked through that production issue together was excellent” reinforces the behaviour you actually want. Individual recognition has its place, but it should be tied to team-enabling behaviours — mentoring, unblocking others, sharing knowledge — not solo heroics.

As for what to base the conversation on practically, keep it simple. For each person, maintain a running list of observations under three headings: things they are doing well, things they need to work on, and situations where you were not sure what to make of their contribution. Review your notes before any one-to-one. Share them openly. Ask the person if they see it the same way. The best feedback conversations are ones where the person mostly agrees with your assessment because nothing in it is a surprise.

The environment is the assessment

On the “did I hire the right people” question, it is worth reframing it entirely. The better question is “have I created an environment where good people can do good work and where struggling people can improve?” If you have strong collaborative practices in place, meaning pairing, mobbing, frequent code review as a learning exercise rather than a gate, continuous integration, and short feedback loops, then you will find that most people rise to the level of the environment. The ones who genuinely cannot are not usually a mystery. They become visible quite quickly when the work is transparent.

This is not a new idea. W. Edwards Deming, whose work on quality management became foundational to both Agile and DevOps, argued decades ago that the system, as designed by leaders, is almost always to blame for problems — not the individual working within the system. Deming saw the manager’s role as improving the system in which people work, not judging the people and leaving the system untouched. His insight that “mistakes typically come from bad systems, not bad workers” has been directly supported by the Accelerate research, which identified 24 organisational capabilities — not individual traits — as the drivers of software delivery performance.

Pink’s motivation framework reinforces the point from the individual’s perspective. Metrics-based evaluation systems undermine autonomy by telling people what to optimise for, distort mastery by rewarding the wrong skills, and corrode purpose by making people focus on numbers rather than outcomes. When you build an environment that gives developers genuine autonomy over how they work, supports their drive for mastery through pairing and challenging problems, and connects their work to a purpose they care about, you create the conditions where good people do their best work — and where struggling people either rise to the challenge or become honestly visible.

The real danger is not hiring the wrong person. It is creating an environment where you cannot tell the difference between a good developer in a bad system and a bad developer in a good system. Make the system transparent and the assessment mostly takes care of itself.

That said, environment-first thinking does not mean never addressing individual performance. Some people genuinely underperform even in excellent environments. The point is the order of operations: fix the system first, then assess the individual. If someone is still struggling after you have given them good practices, clear expectations, psychological safety, timely feedback, and opportunities to grow, then you have a genuine individual performance problem — and you have the evidence to address it fairly, because you have been watching the work all along.

Here is what that conversation sounds like when you have done the work: “Over the last three months, I have noticed a pattern. When you paired with Laura on the billing service, she drove all the design decisions and you did not push back on any of them. The migration task I gave you came back without tests, and the team spent two days fixing the issues it caused downstream. In our last three one-to-ones, I asked what you would change about how we work and you said you were not sure. I have given you feedback on each of these as they happened, and I have not seen a change. I want to help you get to where you need to be, but I need to see a shift in the next few weeks, and here is specifically what that looks like.” That is a hard conversation, but it is not an unfair one. Nothing in it is a surprise. Nothing in it is a vibe. Every claim points to something that actually happened, and the person has already heard about each of those moments because you addressed them at the time. That is the payoff of proximity: when the difficult conversation finally has to happen, it is grounded in evidence both of you recognise.

The underlying principle running through all of this is proximity. You cannot evaluate people from a distance. Dashboards, velocity charts, and ticket counts are all ways of trying to understand people without actually being close to the work, and they consistently fail. The managers who truly understand their teams are the ones who stay close to the work itself. Not micromanaging, not controlling, just present and paying attention — while remaining honest about their own biases, deliberate about distributing their attention fairly, and disciplined about grounding their judgements in evidence rather than impressions.

References

Books

• Forsgren, N., Humble, J., Kim, G. (2018). Accelerate: The Science of Lean Software and DevOps. IT Revolution Press.

• Pink, D. (2009). Drive: The Surprising Truth About What Motivates Us. Riverhead Books.

• Larson, W. (2021). Staff Engineer: Leadership Beyond the Management Track.

• Deming, W.E. Out of the Crisis. MIT Press.

Academic and peer-reviewed research

• Bernstein, E. (2012). “The Transparency Paradox: A Role for Privacy in Organizational Learning and Operational Control.” Administrative Science Quarterly, 57(2), 181–216.

• Forsgren, N., Storey, M-A., Maddila, C., Zimmermann, T., Houck, B., Butler, J. (2021). “The SPACE of Developer Productivity: There’s More to It Than You Think.” ACM Queue, 19(1).

• Google re:Work. “Guide: Understand Team Effectiveness” (Project Aristotle).

• Hawthorne Effect research (1924–1932). Western Electric / Elton Mayo.

Industry sources

• Orosz, G. (2023). “Measuring Developer Productivity? A Response to McKinsey.” The Pragmatic Engineer.

• Orosz, G. “Common Performance Review Biases.” The Pragmatic Engineer.

• Orosz, G. “Software Developer Promotions: Advice to Get to That Next Level.” The Pragmatic Engineer.

• Swarmia (2025). “Engineering Metrics Leaders Should Track.”

• DORA. “DORA’s Software Delivery Performance Metrics.” dora.dev.

Organisational research

• Gallup (2020). Employee engagement and feedback frequency research.

• SHRM. Research on recency and central tendency errors in annual appraisals.

• ISACA (2024). “Examining the Risks of IT Hero Culture.”

• Adobe Systems. Continuous check-in programme results.

Behavioural science

• Goodhart, C. “Goodhart’s Law.” Popularised as: “When a measure becomes a target, it ceases to be a good measure.”

• Harvard Business Review (2018). “3 Biases That Hijack Performance Reviews, and How to Address Them.”

For decades, the software industry has been trying to measure developer productivity the way factories measure widget output — and failing spectacularly. From lines of code to story points, from velocity charts to McKinsey frameworks, every attempt to reduce the complex, creative act of software development to a number has produced the same result: perverse incentives, gaming, and surrogation — the nonconscious process by which people forget what the metric was supposed to represent and treat the number itself as the goal. Modern frameworks like DORA and SPACE represent genuine progress, yet they too are routinely misapplied. Now, AI coding assistants have entered the picture, promising to make every developer a “10x engineer.” But the research tells a far more nuanced and troubling story: AI amplifies whatever is already there — good practices and bad metrics alike. This article traces the history of productivity measurement in software, dismantles the most persistent myths, examines what the evidence actually says about AI-assisted development, and offers leadership a fundamentally different way to think about the question. The answer, it turns out, is not a better metric. It is a better question.

1. Minus Two Thousand Lines of Code

In 1982, Bill Atkinson was rewriting QuickDraw’s region calculation engine for the Apple Lisa. He replaced a complex algorithm with a simpler, more general one that ran approximately six times faster. The new code was elegant, compact, and superior in every measurable way. It also happened to be 2,000 lines shorter than what it replaced.

That week, Apple’s managers required every engineer to submit a form reporting how many lines of code they had written. Atkinson wrote -2000 on his form.

Management discontinued the reporting shortly after.

Atkinson believed that “lines of code was a silly measure of software productivity” and that such metrics “only encouraged writing sloppy, bloated, broken code.” He was right — and yet, more than four decades later, the software industry is still searching for the number that captures what a developer is worth. We are still, in essence, asking engineers to fill out the same form. We have just made the form more sophisticated.

The pursuit itself is the problem. And now, with AI generating code at unprecedented speed, we are not getting closer to an answer. We are getting further away, faster.

2. The Graveyard of Bad Metrics

Every era of software engineering has produced its own favourite way to measure developer productivity. Every one of them has failed. Understanding why they failed is more instructive than understanding what they measured.

Lines of Code

The most intuitive metric is also the most thoroughly discredited. Bill Gates reportedly said that “measuring software productivity by lines of code is like measuring progress on an airplane by how much it weighs.” The problems are well-documented: a task requiring 100 lines in C++ might take 10 in Python, making cross-project comparisons meaningless. Refactoring, deduplication, and simplification — the hallmarks of good engineering — all reduce line count. Under an LOC regime, the best work registers as negative productivity.

Martin Fowler put it simply: well-designed code is shorter because it eliminates duplication. Copy-paste programming inflates LOC while degrading design. LOC indicates system size, not value created.

Story Points and Velocity

Story points and velocity were designed as planning tools — ways for teams to forecast how much work they could take on in a sprint. They were never meant to measure productivity. But in organization after organization, they have been weaponized for exactly that purpose.

The problems are predictable. When velocity becomes a target, teams inflate their estimates to appear more productive. Individual teams can easily game the system, and when some members start inflating story points, others notice. From there, it is a short path to broken trust, frustration, and culture damage. Story points focus on effort, not value. A team can complete 50 points of features that nobody uses and score higher than a team that delivers 20 points of work that transforms the business.

Hours Worked

In knowledge work, more hours frequently produce worse outcomes. Context switching after interruptions costs approximately 23 minutes of refocus time per interruption. Attention residue from task switching can reduce cognitive capacity for 10 to 30 minutes. Cal Newport’s research on “deep work” demonstrates that sustained focus periods of 90 or more minutes are necessary for complex problem-solving. Measuring hours measures presence, not thought. And in an industry where the most valuable breakthroughs often happen during a walk or in the shower, presence is a particularly poor proxy for contribution.

Three Concepts That Explain Everything

Three concepts explain why every simple metric fails — and will always fail.

The McNamara Fallacy, named for U.S. Secretary of Defense Robert McNamara, describes a four-step descent into delusion: First, measure whatever can be easily measured. Second, disregard what cannot be easily measured. Third, presume that what cannot be measured is not important. Fourth, declare that what cannot be measured does not exist.

During the Vietnam War, McNamara used enemy body counts as the primary measure of success. When a general suggested adding a factor for the sentiments of the Vietnamese people, McNamara erased it from the report — he could not quantify it. The war was lost despite the metrics showing consistent progress. The software industry repeats this pattern with striking fidelity: we measure commits, pull requests, and deployment frequency while ignoring design quality, knowledge sharing, and whether anyone actually uses what was built.

Goodhart’s Law states: “When a measure becomes a target, it ceases to be a good measure.” In software engineering, this manifests everywhere: developers rush deployments to meet frequency targets, producing unstable code; teams close easy tickets to inflate resolution rates; engineers write verbose code to increase line counts; managers ship unnecessary features to hit delivery targets. The metric improves. The system degrades.

Surrogation is the most insidious of the three, because unlike gaming, it is invisible to the person it affects. Formally defined by Choi, Hecht, and Tayler in a 2012 paper in The Accounting Review, surrogation describes the nonconscious process by which people stop treating a metric as a proxy for a goal and start treating the metric as the goal. The metric does not just become a target — it replaces the thing it was supposed to represent in people’s minds.

The mechanism is what psychologists call attribute substitution: when the thing you actually care about (developer productivity, code quality, customer satisfaction) is abstract and hard to observe, and the metric (velocity, deployment frequency, NPS score) is concrete and immediately available, your brain quietly swaps one for the other. You do not notice the substitution. You genuinely believe you are pursuing the original goal when you are, in fact, optimising a number.

A person affected by Goodhart’s Law may know they are gaming. A person affected by surrogation does not know they are substituting. And critically, research by Black, Meservy, Tayler, and Williams (2021) demonstrated that you do not even need to tie compensation to a metric for surrogation to occur. Simply tracking and reporting a number is enough. The mere existence of the metric triggers the substitution.

Harris and Tayler, writing in Harvard Business Review in 2019, identified three conditions that create surrogation risk: the strategic objective is abstract, the metric is concrete and visible, and people accept the measure as representing the goal. Software engineering meets all three conditions for virtually every metric it uses.

These three concepts are not bugs in specific metrics. They are features of the relationship between measurement and human cognition. The McNamara Fallacy explains why we ignore what we cannot count. Goodhart’s Law explains why people game what we do count. Surrogation explains why we forget what we were counting in the first place. Any single metric used for evaluation will eventually be gamed, surrogated, or both. This is not cynicism — it is a predictable consequence of how human minds interact with numbers.

3. The Modern Frameworks: Progress and Pitfalls

The good news is that the industry has produced genuinely better thinking about developer productivity. The bad news is that better thinking is routinely applied in the same old ways.

DORA Metrics

The four DORA metrics — Deployment Frequency, Lead Time for Changes, Mean Time to Recovery, and Change Failure Rate — were introduced by Dr. Nicole Forsgren, Jez Humble, and Gene Kim in Accelerate (2018). They represent a significant leap forward because they measure outcomes (how effectively software reaches users) rather than outputs (how much stuff developers produce).

But DORA’s creators explicitly warn against using these metrics for team-by-team comparison. Yet that is precisely what happens: leadership teams benchmark teams on them, comparing deployment frequency between a mobile app team and a web service team as if the numbers were comparable. DORA metrics do not capture developer satisfaction, cognitive load, code quality, technical debt, or business value. A team can score “elite” on all four metrics while building a product that nobody wants.

The 2025 DORA Report acknowledged these limitations by abandoning the low/medium/high/elite performance tiers entirely, replacing them with seven team archetypes — from “harmonious high-achievers” to “legacy bottleneck” teams — reflecting a more nuanced understanding that performance is contextual and multi-dimensional.

SPACE Framework

SPACE, developed in 2021 by Nicole Forsgren, Margaret-Anne Storey, and colleagues at Microsoft Research, explicitly addresses the multi-dimensional nature of productivity across five dimensions: Satisfaction and well-being, Performance, Activity, Communication and collaboration, and Efficiency and flow. Its key insight is powerful: “Productivity cannot be reduced to a single dimension (or metric!). Only by examining a constellation of metrics in tension can we understand and influence developer productivity.”

Forsgren herself warned against the most common misuse: “One of the most common myths — and potentially most threatening to developer happiness — is the notion that productivity is all about developer activity, things like lines of code or number of commits. More activity can appear for various reasons: working longer hours may signal developers having to ‘brute-force’ work to overcome bad systems or poor planning.”

Developer Experience as a Lens

A more recent strand of thinking shifts the focus away from what developers produce and toward what developers experience. The core insight is that three dimensions shape how effectively developers can work: feedback loops (how quickly they get information about their work), cognitive load (how much mental effort their environment demands), and flow state (how often they can achieve sustained, focused work). When these conditions improve, outcomes follow. When they degrade, no amount of measurement or exhortation will compensate.

The McKinsey Debacle

In August 2023, McKinsey published “Yes, you can measure software developer productivity,” claiming their framework was already in use at nearly 20 companies. The response from the engineering community was swift and devastating — the strongest collective rebuttal in recent memory.

Kent Beck, the creator of Extreme Programming, called the report “so absurd and naive that it makes no sense to critique it in detail.” But then he critiqued it in detail, because “what they published damages people I care about. I’m here to help geeks feel safe in the world. This kind of surveillance makes geeks feel less safe.”

Dave Farley, co-author of Continuous Delivery, was blunt: “Apart from the use of DORA metrics in this model, the rest is pretty much astrology.” He drew a sharp line between DORA’s evidence-based approach and the rest of McKinsey’s framework — “the difference between astronomy and astrology.”

Beck and Gergely Orosz, in a detailed joint response, argued that the McKinsey framework “only measures effort or output, not outcomes and impact, which misses half of the software developer lifecycle.” They warned that “introducing a kind of framework that McKinsey is proposing is wrong-headed and certain to backfire. Such a framework will most likely do far more harm than good to organizations — and to the engineering culture at companies and the damage could take years to undo.”

Beck shared a cautionary tale from Facebook that perfectly illustrates surrogation in action. The company had introduced developer satisfaction surveys — a reasonable approach. But then managers computed an overall score. Then those scores appeared in performance reviews. Directors pressured managers for better scores. Managers began negotiating with engineers: higher survey scores in exchange for better performance ratings. The result was worse organisational outcomes despite improved metrics. First, surrogation: leadership began treating the score as developer satisfaction, forgetting that it was merely a proxy. Then Goodhart’s Law kicked in: once the surrogate became a target, people gamed it. The metric had consumed the thing it was supposed to measure.

The McKinsey episode matters because it revealed a fault line that runs through every discussion of developer productivity: the tension between what management consulting wants (simple numbers that enable comparison and control) and what software engineering actually is (complex, collaborative, creative knowledge work that resists simplification).

4. Why Software Is Fundamentally Different

The reason every productivity metric fails or gets corrupted is not that we haven’t found the right metric yet. It is that software development is a fundamentally different kind of work than what most measurement frameworks were designed for.

Knowledge Work Is Not Manufacturing

Peter Drucker, who coined the term “knowledge work,” identified the crucial distinction: in manufacturing, the task is defined externally, and the worker’s job is to execute efficiently. In knowledge work, “the task of what to do is controlled by knowledge workers, they own the means of production (knowledge not machines), they decide what methods and steps to use, and they focus on what to do and on the right things — effectiveness.” Drucker considered knowledge-work productivity “the greatest management task of this century, just as making manual work productive was the great management task of the last century.”

The manufacturing metaphor fails for software because output is not standardised (every piece of software solves a different problem), efficiency is not the primary constraint (figuring out what to build matters more than how fast you build it), quality is not independently measurable (a feature that works but nobody needs has negative value), and the work itself is fundamentally creative — requiring “anticipatory imagination, problem solving, problem seeking, and generating ideas.”

Output vs. Outcome

Martin Fowler articulates the core principle: “If a team delivers lots of functionality... that functionality doesn’t matter if it doesn’t help the user improve their activity.” Output is what is produced: features shipped, code written, pull requests merged. Outcome is the change that the output creates: increased revenue, reduced support tickets, improved user satisfaction.

The insight is disarmingly simple: “Your job is to minimize output, and maximize outcome and impact.” The best solution is often the smallest change, the feature not built, the code deleted. Under any output-based metric, these acts of productive restraint are invisible or actively penalised.

Fowler dismisses the objection that outcomes are hard to measure: “We are very good at measuring financial outcomes.” The problem is not that outcomes cannot be measured, but that organisations prefer the comfort of output metrics because they are immediate and controllable — even when they measure the wrong thing.

The Invisible Work Problem

Charity Majors, CTO of Honeycomb, offers perhaps the most uncomfortable truth: “Some of the hardest and most impactful engineering work will be all but invisible on any set of individual metrics.”

Consider the work that keeps a team effective: pairing and mobbing that improve quality and spread knowledge across the team — but show up as “two people doing one person’s job” in output metrics. Mentoring that multiplies team capacity — but reduces the mentor’s personal output. Architectural decisions that save months of future work — but take hours that produce no visible deliverable. Cross-team coordination that unblocks others. Documentation that prevents future confusion. Technical debt reduction that improves future velocity but produces zero features.

None of this registers on any output metric. All of it is essential.

Majors pushes further: “Metrics are for easy problems — discrete, self-contained, well-understood problems. The more challenging and novel a problem, the less reliable these metrics will be.” And the most provocative observation: “To the extent you can reduce a job to a set of metrics, that job can be automated away.” If developer productivity could truly be captured in a number, developers would already be obsolete.

The Individual Measurement Trap

Dave Farley puts it plainly: “Measuring software development in terms of individual developer productivity is a terrible idea. Being smart in how we structure and organize our work is much more important than the level of individual genius.”

The harms are well-documented. Individual measurement distorts behaviour: people optimise for their own numbers at the expense of the team. It creates misaligned incentives: short-term outputs over long-term value. It renders collaborative work invisible: the developer who writes little code but dramatically improves team performance through guidance and knowledge sharing is undervalued. And it damages culture: “Monitoring individual performance can cause unnecessary anxiety, even for top-performing contributors. It can also lead to overworking, an overly competitive work culture, and developer burnout.”

5. Enter AI: The Amplifier of Everything

Into this already confused landscape, artificial intelligence has arrived with the promise of making every developer dramatically more productive. The reality is far more complex — and for leaders relying on traditional metrics, far more dangerous.

We Do Not Know What We Think We Know

Studies on AI-assisted coding productivity contradict each other wildly — some report massive speed gains, others find slowdowns and more bugs. But the honest takeaway is not that “it depends.” It is that we simply do not have reliable data yet. Every developer uses AI differently — different tools, different prompting styles, different levels of trust and scrutiny, different types of work. The studies measure wildly different things under wildly different conditions. They are not two sides of a debate. They are measurements of entirely different activities that happen to share the label “coding with AI.”

Until the industry converges on how AI is actually used in practice, the numbers should be treated with deep scepticism. Leaders who cite any of these studies to justify or reject AI investment are building on sand.

What we can observe, however, is a persistent perception gap: developers consistently believe AI is helping them more than it measurably does. This matters for leadership. If developers cannot accurately assess whether AI tools are making them more effective, then the comfortable feedback loop — “we bought the tool, developers say they like it, therefore it’s working” — may be an illusion.

We can also observe quality trends that should concern anyone paying attention. Code duplication is rising. Refactoring is collapsing. Code churn is increasing. Even the CEO of Cursor — a company that sells an AI coding tool — has warned publicly that developers accept AI-generated code “simply because it appears to work, without properly reviewing its structure, logic and long-term impact.” When the people selling the tools are urging caution, it is worth listening.

AI Makes Bad Metrics Worse

Every problem with metrics described in this article — gaming, perverse incentives, the McNamara Fallacy, Goodhart’s Law, surrogation — becomes dramatically worse when AI enters the picture.

A human developer gaming a velocity target has natural friction: there are only so many hours in a day, only so much code a person can write. AI has none of these constraints. If the metric is lines of code, AI will generate mountains of it. If the metric is pull requests merged, AI will create dozens. If the metric is deployment frequency, AI will ship constantly. All while the codebase bloats, technical debt compounds, and the actual product — the thing users need — remains unchanged or gets worse.

AI also deepens surrogation. When a human writes code, reviewers can draw on shared context to assess whether the work actually serves the original goal. AI-generated output lacks that shared context, making it harder to notice the gap between the metric and the thing the metric was supposed to represent. And AI tool adoption metrics are themselves susceptible to surrogation: “AI adoption rate” or “percentage of code generated by AI” can quietly replace the actual goal — effective use of AI to improve engineering outcomes — in leaders’ minds. Adoption becomes the goal, regardless of whether it improves anything.

Goodhart’s Law becomes more dangerous as optimisation power increases. AI is a step-function increase in optimisation power. Applied to bad metrics, it does not merely fail — it fails spectacularly, at speed, and at scale.

The 2025 DORA Report puts it simply: “AI doesn’t fix a team; it amplifies what’s already there.” AI is not a productivity solution. It is a productivity amplifier. And amplifiers do not care what signal they are boosting.

6. What Actually Works

If simple metrics fail, modern frameworks are misapplied, and AI amplifies dysfunction, what should leadership actually do? The evidence points toward a fundamentally different approach — one that requires more organisational maturity but produces genuinely better results.

Measure Experience, Not Output

The most actionable path forward is to stop measuring what developers produce and start measuring the conditions that enable production:

Feedback loops. How quickly do developers get information about their work? How long does a CI pipeline take? How fast do code reviews come back? How soon after deployment do they know if something broke? Every minute of waiting is a minute of lost context, and context — not typing speed — is the primary constraint on developer effectiveness.

Cognitive load. How much mental effort does the development environment demand? How many systems must a developer understand to make a change? How clear is the documentation? How usable are the tools? Teams drowning in complexity do not need productivity metrics. They need simpler systems.

Flow state. How often can developers achieve sustained, uninterrupted focus? Research shows it takes approximately 23 minutes to refocus after each interruption, and minimum effective deep work periods are around 90 minutes. An organisation that interrupts developers every 30 minutes for status meetings and Slack messages is not suffering from a productivity problem. It is suffering from a management problem.

Almost half of tech managers now report that their companies measure developer productivity, developer experience, or both. Google, Microsoft, and Spotify have long relied on developer surveys to understand the conditions their teams work in. The shift is happening — but too slowly, and too often alongside the old metrics rather than replacing them.

Focus on Team Outcomes, Not Individual Metrics

Kent Beck’s recommendation is elegant in its simplicity: focus on “producing at least one customer-facing thing per team, per week.” This is not a metric to be gamed. It is a practice that aligns incentives: the whole team collaborates to deliver something a customer can see and respond to. Value is delivered. Feedback is gathered. The cycle continues.

Gergely Orosz and Abi Noda studied 17 major tech companies and found that “rather than wholesale adoption of frameworks like DORA, leading teams use a mix of org-specific qualitative and quantitative metrics.” The best organisations do not adopt a framework off the shelf. They build their own understanding of what “good” looks like in their specific context.

Use Qualitative Approaches Seriously

Google’s approach to developer productivity measurement reveals a counterintuitive truth: qualitative metrics — “measurements comprised of data provided by humans” — capture what automated systems cannot. Flow state, codebase navigability, technical debt perception, satisfaction — these are real and consequential, and they can only be measured by asking people.

Google’s own analysis of 117 metrics for technical debt found that none were valid indicators. Human judgement about the gap between ideal and actual state proved essential. The numbers, on their own, told the wrong story.

Practical recommendations from the research: start with qualitative baselines to identify opportunities, then deploy targeted quantitative metrics for deeper analysis. Segment by team and persona rather than aggregating company-wide. Prioritise free-text comments — developers suggest improvements and identify gaps that structured questions miss. Use transactional surveys at workflow touchpoints for granular, timely feedback. And above all, act on what you learn. Nothing kills a survey programme faster than asking for feedback and visibly ignoring it.

Think in Systems, Not Individuals

W. Edwards Deming’s insight from 1993 remains foundational: “Left to themselves, system components become selfish, competitive, independent profit centres and thus destroy the system. The secret is cooperation between components toward the aim of the organisation.”

The practical application is straightforward: use Theory of Constraints thinking. Identify the bottleneck in the system and focus improvement efforts there. Individual developer speed is rarely the actual constraint. More often, it is slow code reviews, unclear requirements, flaky CI pipelines, cumbersome deployment processes, or poor cross-team communication. Making developers type faster — whether through training or AI — does nothing to address these systemic bottlenecks.

Use AI Wisely

The evidence does not say “don’t use AI.” It says “use AI with open eyes.” AI coding assistants are genuinely valuable for boilerplate, repetitive tasks, and exploration. They are genuinely dangerous when used uncritically on complex, context-rich work — and when their output is measured by the same broken metrics that have always failed.

The 2025 DORA finding bears repeating: AI amplifies what is already there. Before investing in AI tools, invest in the practices that AI will amplify. Fix the feedback loops, reduce the cognitive load, protect the flow state, clarify the team outcomes. Then introduce AI into a healthy system — where it will make good things better rather than making bad things faster.

7. The Question Behind the Question

When an organisation asks “How do we measure developer productivity?”, it is worth pausing to ask why.

Sometimes the answer is benign: we want to understand where our bottlenecks are so we can remove them. Sometimes it is less benign: we want to identify which engineers to fire. Often, it is anxious: we are spending a lot on engineering and we do not know if we are getting value.

Each of these motivations leads to a different approach. The first calls for systems thinking and understanding what developers actually experience. The second calls for an honest conversation about whether the problem is individual performance or organisational dysfunction. The third calls for outcome measurement — connecting engineering work to business results, accepting that the feedback loop is long, and resisting the temptation to substitute easy output metrics for hard outcome questions.

The real question is not “How productive are our developers?” The real question is: “How do we create the conditions where developers can do their best work?”

This is a harder question. It does not produce a single number. It cannot be answered by a dashboard. It requires leadership to engage with the actual work of software development — its complexity, its creativity, its inherent resistance to simplification.

But it is the right question. And in the age of AI, where the temptation to optimise bad metrics at machine speed has never been greater, asking the right question has never mattered more.

Bill Atkinson understood this in 1982. He made software six times faster by writing 2,000 fewer lines of code. By every metric except the ones that matter, he had a terrible week.

By the ones that matter, it was one of the most productive weeks in the history of software engineering.

References

1. Folklore.org, “Negative 2000 Lines Of Code” — https://www.folklore.org/Negative_2000_Lines_Of_Code.html

2. Martin Fowler, “CannotMeasureProductivity” (2003) — https://martinfowler.com/bliki/CannotMeasureProductivity.html

3. Martin Fowler, “OutcomeOverOutput” — https://martinfowler.com/bliki/OutcomeOverOutput.html

4. Martin Fowler, “Measuring Developer Productivity via Humans” — https://martinfowler.com/articles/measuring-developer-productivity-humans.html

5. Nicole Forsgren et al., “The SPACE of Developer Productivity,” ACM Queue (2021) — https://queue.acm.org/detail.cfm?id=3454124

6. Nicole Forsgren, Jez Humble, Gene Kim, Accelerate (2018)

7. DORA Metrics Guide — https://dora.dev/guides/dora-metrics/

8. DORA Report 2025, Google Cloud — https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report

9. Kent Beck & Gergely Orosz, “Measuring developer productivity? A response to McKinsey,” The Pragmatic Engineer (2023) —

10. Gergely Orosz, “Measuring Developer Productivity: Real-World Examples,” The Pragmatic Engineer —

11. Dave Farley, “What McKinsey got wrong about developer productivity,” LeadDev — https://leaddev.com/process/what-mckinsey-got-wrong-about-developer-productivity

12. Charity Majors, “Questionable Advice: Can Engineering Productivity Be Measured?” — https://charity.wtf/2020/07/07/questionable-advice-can-engineering-productivity-be-measured/

13. Peng et al., “The Impact of AI on Developer Productivity: Evidence from GitHub Copilot,” ArXiv (2023) — https://arxiv.org/abs/2302.06590

14. GitHub Blog, “Research: Quantifying GitHub Copilot’s Impact on Developer Productivity and Happiness” — https://github.blog/news-insights/research/research-quantifying-github-copilots-impact-on-developer-productivity-and-happiness/

15. METR, “Measuring the Impact of Early 2025 AI on Experienced Open-Source Developer Productivity” (2025) — https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/

16. Uplevel Data Labs, “A Data-Driven Look at Gen AI for Coding” (2024) — https://resources.uplevelteam.com/gen-ai-for-coding

17. GitClear, “AI Copilot Code Quality Research 2025” — https://www.gitclear.com/ai_assistant_code_quality_2025_research

18. Fortune, “Cursor CEO Michael Truell warns vibe coding builds ‘shaky foundations’” (2025) — https://fortune.com/2025/12/25/cursor-ceo-michael-truell-vibe-coding-warning-generative-ai-assistant/

19. Peter Drucker, “Knowledge-Worker Productivity: The Biggest Challenge” —

20. Cal Newport, Deep Work: Rules for Focused Success in a Distracted World

21. W. Edwards Deming, The New Economics for Industry, Government, Education (1993)

22. Wikipedia, “McNamara Fallacy” — https://en.wikipedia.org/wiki/McNamara_fallacy

23. Wikipedia, “Goodhart’s Law” — https://en.wikipedia.org/wiki/Goodhart’s_law

24. Scrum.org, “Velocity, the False Metric of Productivity” — https://www.scrum.org/resources/blog/velocity-false-metric-productivity

25. LinearB, “Why Agile Velocity is the Most Dangerous Metric” — https://linearb.io/blog/why-agile-velocity-is-the-most-dangerous-metric-for-software-development-teams

26. Aviator, “Everything Wrong with DORA Metrics” — https://www.aviator.co/blog/everything-wrong-with-dora-metrics/

27. Matt Hopkins, “Goodhart’s Law for AI Agents” — https://matthopkins.com/business/goodharts-law-ai-agents/

28. InfoWorld, “Software development meets the McNamara Fallacy” — https://www.infoworld.com/article/4010318/software-development-meets-the-mcnamara-fallacy.html

29. Microsoft Research, “The SPACE of Developer Productivity” — https://www.microsoft.com/en-us/research/publication/the-space-of-developer-productivity-theres-more-to-it-than-you-think/

30. ArXiv, “Leveraging Creativity in Software Engineering” — https://arxiv.org/html/2502.03280v1

31. Choi, J., Hecht, G., and Tayler, W.B. (2012). “Lost in Translation: The Effects of Incentive Compensation on Strategy Surrogation.” The Accounting Review, 87(4), 1135-1164.

32. Black, P., Meservy, T., Tayler, W.B., and Williams, J.O. (2021). “Surrogation Fundamentals: Measurement and Cognition.” Journal of Management Accounting Research, 34(1), 9-28.

33. Harris, M. and Tayler, W.B. (2019). “Don’t Let Metrics Undermine Your Business.” Harvard Business Review, September-October 2019.

34. Kahneman, D. and Frederick, S. (2002). “Representativeness Revisited: Attribute Substitution in Intuitive Judgment.” In Heuristics and Biases: The Psychology of Intuitive Judgment. Cambridge University Press.

This instinct is understandable, but it misses something fundamental. Conflict doesn’t appear out of nowhere. It surfaces because something underneath is unresolved: competing values, unclear expectations, unmet needs, a power dynamic nobody has named, or simply two people with genuinely different views on what good looks like. Treat the symptom without understanding the cause, and the conflict will come back. Usually louder.

The first shift worth making is conceptual. Conflict is not the opposite of a healthy team. Avoidance is. A team where nobody ever disagrees is not a harmonious team. It’s a team where people have learned that expressing disagreement is unsafe or pointless. That is a much more serious problem, because it’s invisible.

Patrick Lencioni, in his work on team dysfunction, placed absence of trust at the very base of what makes teams fall apart. But trust here doesn’t mean “we all get on well.” It means psychological safety: the shared belief that you can speak honestly without being punished for it. Prof. Amy Edmondson at Harvard has spent decades studying this, and the research is consistent. Teams with high psychological safety don’t avoid conflict. They have more of it. What they’re better at is making it productive.

So the question isn’t how to remove conflict from a team. The question is how to raise the quality of it.

The three patterns underneath most team conflicts

Underneath most team conflicts, if you look closely enough, you’ll find one of a small number of recurring patterns.

The first is competing legitimate goods. Two people are both right, but they’re optimising for different things. The engineer wants to do it properly; the product manager wants to ship it now. The team lead wants consistency; the senior developer wants autonomy. Neither of them is wrong. They’re just prioritising different values that are genuinely in tension. Aristotle would have called this a problem of phronesis, practical wisdom: knowing not just what is good, but how to navigate situations where several good things pull in different directions. Resolving this kind of conflict requires surfacing both sets of values explicitly and deciding together which one takes precedence in this context. That’s not a compromise. It’s a choice made consciously.

The second pattern is attribution asymmetry. We judge ourselves by our intentions, but we judge others by their behaviour. I was blunt because I was under pressure. You were blunt because you don’t respect me. This cognitive bias, well documented in social psychology, turns interpersonal friction into moral judgement faster than almost anything else. Once someone has been labelled as difficult, dismissive, or political, the label tends to stick, and the actual content of the disagreement gets buried under a layer of personal narrative. The antidote is perspective-taking: a genuine attempt to understand what the other person is trying to accomplish, and what pressures they’re navigating. This sounds obvious. It’s surprisingly rare in practice.

The third pattern is structural conflict. This is the one teams most often misdiagnose as personal. Two people seem unable to get along, but actually their roles are set up in a way that makes them adversarial. They’re rewarded for different things, they answer to different people, and the organisation has never clearly decided who owns what. Chris Argyris called this “organisational defensive routines”: the ways in which the system itself generates stress and then buries the evidence. You can coach both individuals to within an inch of their lives and it won’t help, because the conflict is in the structure, not in the people.

Behind every accusation, there is an unmet need

Marshall Rosenberg’s Nonviolent Communication framework offers one of the most practically useful lenses for working through interpersonal conflict. Its core insight is that behind every accusation, there is an unmet need. When someone says “you never listen to me,” they’re not really talking about listening. They’re expressing something about recognition, respect, or belonging. When someone says “this team has no standards,” they’re usually expressing a need for quality, predictability, or professional pride.

NVC asks you to separate observations from interpretations, and to connect behaviour to feelings and needs rather than to character judgements. It sounds clinical when described, but in practice it does something powerful: it shifts conversations from blame to need, which is the only terrain on which genuine resolution becomes possible. You can argue forever about who did what. It’s much harder to dismiss someone’s need once it’s been clearly named.

The philosopher Jürgen Habermas wrote about what he called the ideal speech situation: a conversation in which all parties have equal standing, speak honestly, and are open to being persuaded by the better argument rather than by power or status. It’s utopian, obviously. But it’s useful as a direction to aim in. The question it generates is practical: what would need to be true for this conversation to be one where the best idea wins, rather than the loudest voice or the highest rank?

The drama triangle and why nobody moves forward

The drama triangle, originally described by Stephen Karpman in the 1960s, is worth knowing because it describes a pattern that plays out in teams with almost mechanical regularity. It involves three roles: the persecutor, the victim, and the rescuer. They seem like fixed identities, but they’re not. People shift between them constantly, often within a single conversation. The manager who’s trying to protect their team member (rescuer) becomes the person creating dependency (persecutor). The colleague who feels unfairly treated (victim) becomes the one sabotaging decisions (persecutor). The rescuer feels righteous; the victim feels justified; the persecutor feels either powerful or vindicated. Nobody is moving forward.

The way out of the drama triangle, according to David Emerald’s reframing of it, is to shift from reactive to creative: from “what’s wrong and who’s to blame” to “what do we want and what can we do.” This isn’t about being positive. It’s about redirecting agency. Conflict locks people into a past-oriented, causal story. Resolution requires a future-oriented, intentional one.

The leader’s real job is not to resolve conflict but to contain it

The leader’s role in all of this is often misunderstood. Many leaders believe their job is to resolve conflicts: to step in, assess the situation, and hand down a decision. This sometimes works in the short term. What it teaches the team is that conflict is the leader’s problem to solve, not their own capacity to develop. Over time, it creates a team that escalates rather than resolves, and a leader who is permanently stuck in the middle.

A more useful frame, drawn from systemic coaching and family systems theory, is the leader as container. The job is not to eliminate tension but to create conditions in which the team can sit with tension long enough to understand it. This requires a specific kind of confidence: the ability to resist the pull towards premature resolution. Ronald Heifetz at Harvard Kennedy School distinguishes between technical problems, which have known solutions that someone with expertise can apply, and adaptive challenges, which require the people involved to change how they think. Most team conflicts are adaptive. They can’t be solved by the leader implementing the right answer. They require the team to work through something together.

This is why the most important thing a leader can do in a moment of conflict is often not to solve it, but to slow it down. Ask questions rather than provide answers. Name what’s happening without taking sides. Make the implicit explicit. Create enough safety for people to say what they actually think.

Conflict prevention happens in ordinary moments, not just critical ones

There’s a strand of sociology, particularly in the work of Randall Collins on interaction ritual chains, that points to something teams rarely discuss: the emotional energy in a room. When people feel seen, included, and engaged, they generate positive emotional energy that makes collaboration easier. When they feel marginalised, ignored, or dismissed, they generate negative emotional energy that makes everything harder. This isn’t soft. It’s how groups actually work. Social identity theory, developed by Henri Tajfel and John Turner, shows how quickly people sort themselves into in-groups and out-groups, and how much of team conflict is actually about belonging rather than the ostensible subject of the disagreement.

What this means practically is that a lot of conflict prevention happens not in the difficult moments but in the ordinary ones. Whether you notice someone’s contribution in a meeting. Whether you ask about someone’s view before the decision is made. Whether you check in with the quieter members of the team, not just the loudest. These small acts of inclusion or exclusion accumulate over time and determine whether people feel invested in the team’s success or quietly estranged from it.

When conflict genuinely cannot be resolved

None of this is to say that all conflicts can be resolved by better communication and more empathy. Sometimes values are genuinely incompatible. Sometimes people are acting in bad faith. Sometimes the problem really is that two people cannot work together, and the most honest thing to do is acknowledge that rather than keep pretending otherwise.

The Stoics were clear about what lies within our control and what doesn’t. You can change how you communicate, what you invite, what you model, what structures you put in place. You cannot change another person’s willingness to engage. At some point, continued investment in a conflict that the other party is not interested in resolving becomes a form of self-deception.

But that point comes much later than most people think. The vast majority of team conflicts that get written off as irresolvable have actually never been properly named, never had the structural conditions examined, never had someone ask sincerely what each person actually needs. They’ve had plenty of difficult conversations in which two people talked past each other while the real issue stayed underground.

What to actually do

If I had to distil this into something actionable, it would be this.

Start by assuming the conflict is information. Ask what it’s telling you about values, needs, structures, or dynamics that haven’t been made visible yet. Resist the pull towards fast resolution. Create conditions where people can speak honestly without penalty. Separate the people from the positions and try to understand the interests underneath. Name what’s happening, including the things that feel uncomfortable to name. Ask whose voices are missing from the conversation.

And remember that a team capable of navigating conflict well is not a team that has achieved some kind of permanent peace. It’s a team that has developed the muscle to disagree, recover, and keep going together. That muscle is one of the most valuable things a team can have. It doesn’t come from avoiding conflict. It comes from doing it better, over and over, until it stops feeling like a threat and starts feeling like work.

References

Patrick Lencioni — The Five Dysfunctions of a Team (2002)

Amy Edmondson — The Fearless Organization (2018); “Psychological Safety and Learning Behavior in Work Teams,” Administrative Science Quarterly (1999)

Aristotle — Nicomachean Ethics, on phronesis (practical wisdom)

Chris Argyris — Overcoming Organizational Defenses (1990)

Marshall Rosenberg — Nonviolent Communication: A Language of Life (2003)

Jürgen Habermas — The Theory of Communicative Action (1981)

Stephen Karpman — “Fairy Tales and Script Drama Analysis,” Transactional Analysis Bulletin (1968)

David Emerald — The Power of TED (2009)

Ronald Heifetz — Leadership Without Easy Answers (1994); The Practice of Adaptive Leadership (2009, with Linsky and Grashow)

Randall Collins — Interaction Ritual Chains (2004)

Henri Tajfel and John Turner — “An Integrative Theory of Intergroup Conflict,” in The Social Psychology of Intergroup Relations (1979)

Epictetus — Enchiridion, on the dichotomy of control

This is what happens when some teams switch from Scrum to Kanban.

A team struggles with Scrum. Sprint planning feels pointless. Retrospectives produce nothing. Velocity is treated as a performance metric. They estimate in story points. Standups are boring status reports. Someone reads a LinkedIn post about how “sprints are dead,” and the team decides to make the switch. They rip out the sprints, put up a Kanban board, and carry on.

Nothing actually changes. The same dysfunctions are still there, running underneath a different label. The planning is still vague. The feedback loops are still weak. The problems to solve are still unclear. The team has picked up the other jar and is now confidently defending why this jam tastes better. It’s the same jam. They just can’t tell, because they never understood what they were tasting in the first place.

This article is about why that happens. It’s about what agile frameworks actually do, what most people get wrong about Scrum, what most people get wrong about Kanban, and why the conversation about switching between them is almost always asking the wrong question.

Frameworks are mirrors, not medicines

Here’s the most important thing I can tell you about any agile framework: it doesn’t fix your problems. It makes your problems visible.

A sprint that feels painful isn’t painful because of the sprint. The sprint is showing you something. Maybe your team can’t finish anything in two weeks because the work isn’t broken down properly. Maybe planning feels like fiction because nobody actually understands the requirements. Maybe the demo is embarrassing because the team built something nobody asked for. These are not sprint problems. These are organisational problems, team problems, requirements problems. The sprint is the mirror.

When you remove the mirror, you don’t fix the dysfunction. You just stop seeing it.

This is the role agile frameworks play. Scrum’s ceremonies create regular, unavoidable moments where reality has to be confronted. Did we build the right thing? Are we improving? Can we actually finish what we started? If the answers are uncomfortable, the instinct is to blame the framework. But the framework is just the thing that forced you to ask.

Kanban does the same thing, by the way. A Kanban board with a massive pile of work-in-progress is a mirror showing you that your team can’t say no and can’t finish things. WIP limits that keep getting overridden are a mirror showing you that your organisation doesn’t respect capacity constraints. The question is never “which mirror do I want?” The question is “am I willing to look at what the mirror shows me?”

Most teams that switch from Scrum to Kanban aren’t choosing a better mirror. They’re choosing a less confrontational one. Sprints force you to face reality every two weeks whether you like it or not. Kanban is gentler about it. That’s not always an advantage.

Everything you think you know about Scrum is probably wrong

I’ve coached teams for years, and the number of misconceptions about Scrum is staggering. Most teams that say they’re “doing Scrum” are doing something else entirely, and most teams that say Scrum doesn’t work have never actually tried it. Let me go through the big ones.

Sprints are mini-waterfalls. This is the most common misconception and it’s completely wrong. A sprint is not “two weeks to build what we planned.” A sprint is a timebox for learning. The Sprint Goal gives direction, but the team is expected to discover things along the way, adjust their approach, and deliver the most valuable increments they can. If your sprints feel like waterfall, the problem is how you’re running them, not the concept itself.

Story points are part of Scrum. They’re not. Read the Scrum Guide. Story points appear nowhere in it. Story points are a widespread practice that became associated with Scrum through cultural osmosis, but Scrum doesn’t prescribe any estimation technique. If your story point discussions are wasting hours every sprint, that’s a choice your team made, not something Scrum imposed on you.

Velocity is a Scrum metric. It’s not. It’s not in the Scrum Guide at all. Velocity became widespread through tools like Jira and through management’s desire for predictability metrics. In practice, it’s an antipattern. It incentivises gaming, it conflates output with value, and it gives the illusion of measurement where no real measurement exists. Scrum asks teams to forecast what they can accomplish in a sprint. It doesn’t say how. If your organisation is using velocity as a performance measure or a commitment mechanism, that’s a dysfunction your organisation created, not something Scrum asked for.

You can’t change direction during a sprint. Wrong. The Sprint Goal provides focus, but the Scrum Guide explicitly says the scope of the sprint can be clarified and renegotiated with the Product Owner as more is learned. If something urgent comes up on Monday, nothing in Scrum prevents you from addressing it. If your team feels locked in, the problem is your interpretation, not the framework.

The daily standup is a status report. It’s not. The Daily Scrum is a planning event. Its purpose is for the developers to inspect progress toward the Sprint Goal and adapt their plan for the next 24 hours. It’s not “tell the Scrum Master what you did yesterday.” If that’s what it’s become, you’ve turned a collaborative planning session into a reporting ceremony. The framework didn’t do that. You did.

The Scrum Master is a project manager. The Scrum Master has no authority over the team. They don’t assign work, they don’t track progress, they don’t manage resources. They serve the team by helping remove impediments and by coaching the organisation on Scrum. If your Scrum Master is acting as a project manager, you don’t have a Scrum Master. You have a project manager with a different title.

Retrospectives are blame sessions or wastes of time. If your retrospectives produce nothing, the problem is psychological safety, facilitation, or follow-through. The retrospective exists so the team can inspect how the last sprint went and make concrete improvements. If nobody speaks honestly, that’s a trust problem. If actions are identified but never implemented, that’s a commitment problem. The retrospective is the mirror showing you those problems exist.

Scrum requires a specific team structure. Scrum requires the team to be cross-functional, meaning the team as a whole has all the skills needed to deliver. It does not mean everyone does everything. A team can have specialists. The requirement is that the team doesn’t depend on external groups to complete their work.

Scrum is rigid and prescriptive. Scrum is actually a minimal framework. The Scrum Guide is 13 pages long. It defines three roles, five events, and three artefacts. Everything else, including story points, velocity, burndown charts, task boards, estimation techniques, and technical practices, is something teams and organisations have added on top. Most of the rigidity people complain about isn’t Scrum. It’s the layers of process their organisation built around Scrum.

Scrum doesn’t allow for technical excellence. This one is particularly frustrating. Scrum doesn’t prescribe technical practices because it’s a project management framework, not an engineering framework. But nothing in Scrum prevents you from doing test-driven development, pair programming, continuous integration, refactoring, or any other XP practice. In fact, the Scrum Guide explicitly states that the Scrum Team is expected to maintain a high-quality Definition of Done. If your Scrum implementation ignores technical quality, you’ve chosen to do that.

The Product Backlog is a list of requirements. The Product Backlog is an ordered list of what might be needed to improve the product. “Might” is doing a lot of work in that sentence. It’s not a specification document. It’s a living, evolving artefact that reflects current understanding. If your backlog is being treated as a fixed requirements list, you’ve lost the plot.

Burndown charts are required. Not in the Scrum Guide. Use them if they help. Don’t use them if they don’t. They’re a tool, not a mandate.

Sprints prevent continuous delivery. This is completely false and I’ll address it in detail later. The sprint is a planning and learning boundary, not a release boundary. You can deploy as often as you like within a sprint.

The common thread in all of these misconceptions is that teams confuse the accretions of corporate Scrum with actual Scrum. They rebel against story point negotiations, velocity tracking (an antipattern that Scrum never asked for), burndown charts, and rigid sprint commitments, and they think they’re rebelling against Scrum. They’re not. They’re rebelling against a cargo cult version of Scrum that their organisation created.

The six practices of Kanban and why you must respect them

Here’s where things get ironic. Most teams that switch from Scrum to Kanban because Scrum felt “too prescriptive” end up doing Kanban in a way that ignores most of what Kanban actually requires.

The Kanban Method, as defined by David Anderson, has six core practices. They’re not optional. They’re not suggestions. If you’re ignoring them, you’re not doing Kanban. You’re just not doing Scrum.

Visualise the workflow. This means making the entire flow of work visible, from the moment something is requested to the moment it’s delivered. Not just “To Do, In Progress, Done.” The real workflow, with all its stages, queues, and handoffs. Most teams put up a three-column board and call it Kanban. That’s not visualising your workflow. That’s hiding your workflow behind a simplified picture.

Limit work in progress. This is the single most important practice in Kanban, and it’s the one most teams ignore. WIP limits exist because starting many things and finishing few things is the primary cause of slow delivery. When you limit WIP, you force the team to finish things before starting new ones. This creates flow. Without WIP limits, you don’t have Kanban. You have a to-do list on a wall. And here’s the critical part: WIP limits must be respected. If your WIP limit is 3 and you have 5 items in progress, you don’t have a WIP limit. You have a decoration. The moment WIP limits become negotiable, you’ve lost the core mechanism that makes Kanban work.

Manage flow. Kanban requires you to actively monitor and manage the flow of work through the system. This means measuring lead time, cycle time, throughput, and identifying bottlenecks. It means paying attention to where work gets stuck and taking action to unblock it. If you’re not measuring flow, you have no idea whether your system is improving or degrading. You’re flying blind.

Make process policies explicit. This means writing down the rules. When is an item ready to be pulled into the next stage? What does “done” mean? Who can override a WIP limit, and under what conditions? Most teams that adopt Kanban operate on implicit assumptions that nobody has agreed to. This leads to exactly the kind of confusion and conflict they were trying to escape by leaving Scrum.

Implement feedback loops. Kanban specifies several feedback loops, including daily standup meetings, service delivery reviews, operations reviews, and risk reviews. Yes, you read that correctly. Kanban has regular meetings. If you switched to Kanban because you wanted fewer meetings, you’ve misunderstood the method. The feedback loops in Kanban serve the same purpose as the events in Scrum: they force you to inspect reality and adapt.

Improve collaboratively, evolve experimentally. Kanban emphasises continuous improvement through small, incremental changes based on evidence. This means you need retrospectives, or something functionally equivalent. You need to regularly examine your process, identify problems, try experiments, and measure results. If you switched from Scrum to Kanban and stopped doing retrospectives, you’ve removed your improvement mechanism.

Do you see the irony? Teams abandon Scrum because they find it too structured, then adopt Kanban while ignoring the structures that make Kanban work. The result is a team with no sprints, no WIP limits, no flow metrics, no explicit policies, no feedback loops, and no improvement process. That’s not Kanban. That’s anarchy with a board on the wall.

Why switching without understanding solves nothing

The core fallacy in the “Scrum to Kanban” narrative is the assumption that the framework is the problem. It almost never is. The problems are usually one or more of the following:

The organisation treats agile as a project management methodology rather than a product development approach. Work is pushed onto teams rather than pulled. Success is measured in output rather than outcomes. Management wants predictability but won’t invest in the conditions that create it. Teams don’t have the skills, autonomy, or psychological safety to self-organise. Requirements are vague and nobody has the authority or willingness to clarify them. Technical debt has accumulated to the point where every change is slow and risky. Dependencies between teams create bottlenecks that no process framework can solve.

None of these problems are caused by sprints. And none of them are solved by removing sprints. If you switch from Scrum to Kanban while carrying all of this baggage, you’ll get the same results with a different board layout.

The honest conversation isn’t “should we use Scrum or Kanban?” It’s “what are the actual problems we need to solve, and which practices help us solve them?”

Scrum and Kanban are not opposites

The framing of Scrum versus Kanban is fundamentally wrong. They’re not competing philosophies. They’re complementary sets of practices that address different aspects of the same challenge: how to deliver valuable software in the presence of uncertainty.

Scrum provides a cadence. Regular moments for planning, review, and adaptation. A timebox that creates urgency and focus. Defined roles that clarify responsibility.

Kanban provides flow management. Visualisation of work. WIP limits that prevent overload. Metrics that show whether your system is healthy.

There is nothing stopping you from using both. In fact, the most effective teams I’ve worked with do exactly that. They use sprints as planning and learning cycles while also applying WIP limits within those sprints. They visualise their workflow on a Kanban board while maintaining the Scrum cadence of Sprint Reviews and Retrospectives. They measure flow metrics like cycle time and throughput alongside sprint-based metrics. They treat the sprint as a feedback loop and the Kanban board as a diagnostic tool.

This integration is sometimes called Scrumban, and while the name is clumsy, the idea is sound. Take the useful parts of both. Timeboxes and WIP limits. Sprint Reviews and flow metrics. Retrospectives and explicit policies. The goal is not to pick a camp. The goal is to assemble a set of practices that helps your team deliver effectively.

The question is never “Scrum or Kanban?” The question is “what combination of practices makes our specific problems visible and gives us the feedback we need to improve?”

Continuous delivery works with both, and most teams do neither

One of the strangest arguments in the “sprints are dead” discourse is the idea that sprints prevent continuous delivery. This betrays a fundamental misunderstanding of what a sprint is.

A sprint is a planning boundary, not a release boundary. The Scrum Guide says the increments must be usable by the end of the sprint. It does not say you can only release at the end of the sprint. You can deploy to production every day, multiple times a day, within a sprint. Many teams do. In fact, the Scrum Guide encourages readers to do that: “Multiple Increments may be created within a Sprint. The sum of the Increments is presented at the Sprint Review thus supporting empiricism. However, an Increment may be delivered to stakeholders prior to the end of the Sprint. The Sprint Review should never be considered a gate to releasing value.”

Continuous delivery is a technical capability. It requires automated testing, continuous integration, a deployment pipeline, feature toggles, and an architecture that supports independent deployment. These are engineering practices. They’re independent of whether you use Scrum, Kanban, or anything else.

If your team can’t do continuous delivery, switching from Scrum to Kanban won’t change that. You’ll still have the same manual testing, the same merge conflicts, the same deployment bottlenecks, and the same fear of releasing. Kanban doesn’t give you a deployment pipeline. It doesn’t write your tests. It doesn’t decouple your architecture.

Conversely, if your team can do continuous delivery, Scrum doesn’t prevent it. Deploy whenever you want. The sprint planning still gives you a rhythm for deciding what to work on. The Sprint Review still gives you a moment to inspect what was delivered. The retrospective still gives you a moment to improve.

Continuous delivery is orthogonal to your process framework. It’s an engineering practice, and it works with any framework you choose, provided you invest in the technical foundations that make it possible.

AI changes the speed of writing code, not the nature of risk

The latest twist in the “sprints are dead” argument is AI. The claim goes something like this: AI generates code so fast that delivery is no longer the bottleneck, so we don’t need sprints anymore. We need “context” and “discovery” instead.

Let me be direct about this. This argument confuses writing code with delivering software, and it confuses speed with risk.

Delivery has never been primarily about typing. The time spent writing code has always been a small fraction of the total cost of delivering software. Most of the time goes to understanding what needs to be built, designing a solution, testing it, integrating it with existing systems, deploying it, monitoring it, and supporting it in production. AI can accelerate the coding part. It doesn’t eliminate the rest.

But more importantly, the fundamental challenge of software development hasn’t changed. It’s still risk. Will this feature solve the user’s actual problem? Will it work correctly under real conditions? Will it integrate with existing systems without breaking anything? Will it scale? Will it be maintainable? Will it be secure? Will it comply with regulations?

None of these risks are reduced by generating code faster. If anything, they’re amplified. The faster you can generate code, the faster you can build the wrong thing. The faster you can introduce defects. The faster you can accumulate technical debt. Speed without feedback is not a capability. It’s a hazard.

This is exactly why iterative development exists. Not because coding is slow, but because understanding is incomplete. You build a small thing, put it in front of users, learn from the feedback, and adjust. The speed of the coding step is largely irrelevant to this cycle. What matters is the speed of learning.

AI doesn’t change the need for feedback loops. AI doesn’t change the need for WIP limits. AI doesn’t change the need for regular retrospection. AI doesn’t change the need for explicit process policies. AI doesn’t change the need to manage flow. AI doesn’t change the fact that the biggest risk in software development is building something nobody wants, or building something that doesn’t work in production, or building something that can’t be maintained.

What AI does is make the consequences of poor practices worse. If your feedback loops are weak, you’ll now generate wrong solutions faster. If your requirements process is broken, you’ll now build the wrong thing in minutes instead of weeks. If your testing is inadequate, you’ll now deploy defective code more frequently. AI amplifies whatever practices you already have, good or bad.

The teams that will benefit most from AI are the teams that already have strong practices: clear requirements processes, fast feedback loops, comprehensive automated testing, continuous integration, and the discipline to validate before they ship. These are the teams that can use AI to accelerate the coding step without introducing additional risk, because the rest of their process catches the errors.

The teams that will suffer most are the teams that think AI means they can skip the hard parts. The teams that think “context” replaces testing. The teams that think discovery can be done entirely upfront instead of continuously. The teams that think generating code fast means delivering value fast.

Risk management in software development is the same problem it’s always been. You reduce risk through short feedback cycles, small batch sizes, continuous validation, and close collaboration with the people who will use the software. Sprints do this. Kanban’s WIP limits and flow management do this. Both frameworks, when properly implemented, are risk management tools.

AI doesn’t change what risk is. It just changes how fast you can create it.

The real question

If you’re struggling with Scrum, the answer is not to switch to Kanban. If you’re struggling with Kanban, the answer is not to switch to Scrum. The answer is to look honestly at what the framework is showing you and address the real problems.

Is your planning painful because nobody understands the requirements? Fix the requirements process. Are your retrospectives useless because people don’t feel safe? Build psychological safety. Is your delivery slow because of technical debt? Invest in engineering practices. Are your WIP limits constantly overridden because management keeps pushing work in? Have an honest conversation about capacity.

The framework is the mirror. The dysfunction is yours. Changing the mirror won’t change what it’s reflecting.

Stop debating frameworks. Start fixing the problems the frameworks are showing you.

If you’ve ever sat through a planning session where the team spent twenty minutes debating whether something is a 5 or an 8, you’ve experienced the dysfunction firsthand. That time could have been spent understanding the work, identifying risks, or actually delivering something.

This article is a practical guide for teams who want to move beyond story points toward planning approaches that are simpler, faster, and grounded in reality rather than collective guessing.

Why story points fail

The theory behind story points sounds reasonable: estimate relative complexity rather than time, use historical velocity to forecast, and avoid the trap of conflating estimates with commitments. In practice, this rarely works as intended.

The first problem is that story points mean different things to different people, even within the same team. One developer thinks about complexity. Another thinks about effort. A third is secretly converting to hours in their head and then picking a Fibonacci number. When you average these different mental models together, you don’t get wisdom of crowds. You get noise.

The second problem is that velocity, the sum of story points completed per sprint, is treated as if it were a stable measure when it isn’t. Velocity fluctuates based on who’s on holiday, how many meetings interrupt the sprint, whether the work was estimated by the same people who did it, and countless other factors. Teams often respond to this instability by adding more process: re-estimation, calibration sessions, reference stories. None of this makes the underlying measure more meaningful.

The third problem is that story points create perverse incentives. When velocity becomes a performance metric, teams unconsciously inflate their estimates. A 3 becomes a 5. An 8 becomes a 13. Velocity goes up, but throughput stays the same. Everyone pretends not to notice.

The final problem is opportunity cost. The time spent estimating is time not spent understanding, slicing, or delivering. A team that spends two hours per sprint on estimation and calibration is spending over a hundred hours per year on an activity that doesn’t move the product forward.

What actually predicts delivery

If story points don’t work, what does? The answer is surprisingly simple: count things.

Throughput, the number of items a team completes per unit of time, is a far more stable and useful measure than velocity. This works because throughput is based on observation rather than prediction. You’re not asking “how hard do we think this will be?” You’re asking “how many things did we actually finish?”

For throughput to be a reliable predictor, one condition must be met: the items being counted need to be roughly similar in size. This doesn’t mean identical, just within the same order of magnitude. If most of your work takes one to three days to complete, counting items gives you a useful forecast. If your backlog contains a mix of half-day tasks and three-week epics, counting won’t tell you much.

This is where the real skill comes in. The shift from story points to throughput-based planning is really a shift from estimating to slicing. Instead of asking “how big is this?” you ask “how do we make this small enough that size doesn’t matter?”

The art of slicing work small

Slicing is the most valuable planning skill a team can develop, and it’s almost entirely ignored in mainstream agile training. The ability to take a large, ambiguous piece of work and break it into thin vertical slices that each deliver value is what separates high-performing teams from the rest.

A good slice has three properties. First, it delivers something a user or stakeholder can see, use, or give feedback on. Second, it can be completed independently, without waiting for other slices. Third, it’s small enough to finish in a day or two of focused work.

The third property is the critical one for planning purposes. When everything in your backlog is one to three days of work, the difference between items becomes negligible. A two-day item and a three-day item are close enough that counting them as equivalent introduces less error than trying to estimate them separately.

Teams often resist slicing this small because it feels unnatural. “We can’t deliver anything useful in two days,” they say. This is almost never true. What’s actually happening is that the team has become accustomed to thinking in terms of technical tasks rather than user value. A “user story” that says “implement the payment gateway” isn’t a story at all. It’s a technical component. A real slice might be “a customer can pay for a single item using a saved card” or even “a customer sees a payment button that shows a coming soon message.” Both of these deliver something observable. Both can be done in a day or two. Both give you something to learn from.

Learning to slice well takes practice, and the best way to practice is to do it together as a team. Every time someone brings a large item to planning, treat it as an opportunity to slice collaboratively. Ask: what’s the smallest thing we could deliver that would let us learn something? What could we ship that a user would actually notice? What’s the riskiest part of this, and how could we test that assumption with a thin slice?

How to run planning without estimates

Once your team has embraced small slices and started tracking throughput, planning becomes remarkably simple.

Before the session, gather your historical data. How many items has the team completed in each of the last eight to ten sprints? Calculate the average and note the range. If you’ve been completing between six and ten items per sprint, with an average of eight, that’s your forecast baseline.

Start the planning session by reviewing the team’s throughput data together. This isn’t about judgement or performance management. It’s about grounding the conversation in reality. “Based on our history, we typically complete around eight items per sprint. Sometimes it’s six, sometimes it’s ten. Let’s plan accordingly.”

Then work through the backlog in priority order. For each item, ask three questions. Do we understand what done looks like for this? Is this small enough to complete in a couple of days? Are there any blockers, dependencies, or risks we need to address?

If the answer to the second question is no, stop and slice. This is the most important part of the session. Don’t let large items into the sprint. Every large item is a forecast risk and a flow impediment.

Keep pulling items until you’ve reached your typical throughput number. If your average is eight, stop at eight or perhaps nine. Resist the temptation to overcommit because this sprint “feels different.” It doesn’t. The whole point of using historical data is to protect you from optimism bias.

That’s it. No poker. No Fibonacci. No debates about whether complexity and effort should be weighted differently. Just a focused conversation about understanding the work and making sure it’s small enough to flow.

Answering the objections

When you propose this approach, you’ll face pushback. Here’s how to address the most common objections.

“How will we know if we’ve planned the right amount of work?” You’ll know the same way you know now: by comparing what you planned to what you delivered. The difference is that throughput-based planning gives you an honest forecast based on measurement rather than a confident-sounding number based on guessing. If you consistently complete eight items, planning for eight items is a reasonable bet. If you consistently complete somewhere between six and ten, acknowledge that range rather than pretending you can predict exactly.

“What about items that are genuinely complex and can’t be sliced smaller?” This is almost always a failure of imagination rather than a hard constraint. I’ve worked with teams building safety-critical systems, complex financial products, and intricate distributed architectures. In every case, we found ways to slice work small. The technique varies depending on context, but the principle holds: there’s always a thinner slice that still delivers something real. If you truly cannot slice something smaller, that’s a signal that you don’t yet understand the work well enough. Do a spike. Build a prototype. Run an experiment. Don’t commit to delivering something you can’t decompose.

“Stakeholders expect velocity reports and burn-down charts.” This is a change management challenge, not a technical one. Most stakeholders don’t actually care about velocity. They care about predictability: when will this be done, and can we count on that date? You can answer those questions more honestly with throughput data and cycle time distributions than with velocity. Have a conversation with your stakeholders about what information they actually need and why. Often they’re relieved to stop pretending the current reports mean something.

“Different team members work at different speeds, so counting items doesn’t account for who picks up what.” Neither do story points. You’re measuring team throughput, not individual productivity. Over time, the mix of who does what averages out. If it doesn’t, you have a team design problem that no estimation method will solve.

“We need estimates for roadmap planning and budgeting.” Throughput data supports roadmap planning better than story points. If you have fifty items in the backlog and you complete eight per sprint, you can forecast six to seven sprints of work with appropriate confidence intervals. This is more honest than converting everything to story points and dividing by velocity, which gives you a single number that implies false precision.

Making the transition

If you’re convinced this approach is worth trying, here’s how to introduce it without causing chaos.

Start by gathering data. Even if you’re still using story points, begin tracking throughput alongside velocity. After a few sprints, you’ll be able to show the team how throughput compares in terms of stability and predictability.

Next, invest in slicing skills. Run workshops on vertical slicing. Practice breaking down real backlog items together. Make slicing a core part of your refinement sessions. This is the foundation that makes everything else work.

Then propose an experiment. Suggest trying throughput-based planning for three sprints. Frame it as a learning exercise rather than a permanent change. This reduces resistance and gives sceptics a face-saving way to engage.

During the experiment, facilitate planning sessions that focus on understanding and slicing rather than estimating. Keep the conversations grounded in “is this small enough?” rather than “how big is this?”

After three sprints, retrospect together. Was planning faster? Did the team deliver roughly what they forecast? Did the conversations during planning feel more useful? In my experience, teams rarely want to go back once they’ve experienced the simplicity of this approach.

The deeper shift

Moving away from story points isn’t just a process change. It’s a shift in mindset from prediction to measurement, from estimation to understanding, from big batches to small slices.

This shift has benefits beyond planning. Small slices improve flow, reduce risk, accelerate feedback, and make continuous integration actually possible. Teams that slice well deliver more frequently and learn faster. They spend less time in meetings and more time shipping.

Story points were never the point. The point was always to deliver valuable software sustainably. If your current approach to planning isn’t helping you do that, it might be time to try something simpler.

Further Reading

Software estimation has a terrible track record. Average cost overruns of 189% and time overruns of 222% aren’t anomalies; they’re the norm. But the problem isn’t that the future is unknowable. The problem is that organisations punish honesty about uncertainty.

The core claim: Estimation fails not because prediction is impossible, but because most organisations can’t handle honest answers. They demand single numbers, treat those numbers as commitments, and blame people when reality diverges from the guess. No methodology fixes that.

What doesn’t work: Traditional point estimates pretend certainty where none exists. Story points get gamed the moment they become performance metrics. Even probabilistic forecasting fails if the organisation punishes you for missing the 75th percentile instead of the point estimate.

What actually helps: Track what happens (cycle time, throughput). Present ranges with confidence levels. Keep work items small, genuinely small, using deliberate slicing techniques. Treat estimates as decision-support, not promises. Most importantly, build organisational tolerance for uncertainty. If your culture punishes honest answers, no methodology will save you.

The bottom line: The question isn’t “how do we estimate better?” The question is “how do we build organisations mature enough to handle honest answers about uncertainty?” That’s harder than switching methodologies, but it’s the only thing that actually works.

The terminology trap

In brief: Organisations routinely conflate estimates, targets, and commitments. This vocabulary confusion is where dysfunction begins.

Before examining why estimation fails, we need to untangle a vocabulary problem that poisons most estimation conversations.

Steve McConnell, in Software Estimation: Demystifying the Black Art, identifies three concepts that organisations routinely conflate:

An estimate is an analytical prediction of how long something will take, based on available information. It’s a forecast, not a promise. It should include uncertainty ranges and will change as information improves.

A target is a business goal or desired deadline. It represents what the organisation wants to achieve, independent of whether it’s realistic.

A commitment is a promise to deliver by a specific date. It carries accountability and should only be made when uncertainty has narrowed enough that the risk is acceptable.

The dysfunction begins when these get confused. A developer saying “this looks like maybe six weeks” intends an estimate, a preliminary forecast based on incomplete information. Management hears a commitment. The roadmap records a target. When reality diverges, everyone feels betrayed despite never having agreed on what the original number meant.

This isn’t mere pedantry. McConnell argues that consciously separating these concepts is a prerequisite for healthy estimation conversations. An estimate can inform whether a target is realistic. A commitment should only follow once estimates have been refined sufficiently that the uncertainty band becomes acceptable for the business risk involved.

Most organisational dysfunction around estimation traces back to this confusion. When executives treat preliminary estimates as commitments, or when targets are disguised as “just estimates,” the game is rigged before it starts.

The uncomfortable track record

In brief: The statistics are bad, but often overstated. The real problem isn’t prediction itself; it’s what organisations do with predictions.

Let’s be honest about the numbers, and careful about what they actually mean.

Bent Flyvbjerg’s research on over 16,000 projects found that only 0.5% deliver their promised benefits within budget and timeframe. The Standish Group’s CHAOS Report shows roughly 16% of projects succeed on time, on budget, with all features. About 53% are challenged, meaning over budget, over time, or with fewer features. The remaining 31% are cancelled outright.

These numbers are often cited to damn estimation itself, but we should be careful. Flyvbjerg’s research focuses heavily on large infrastructure and megaprojects: airports, railways, Olympic venues. These domains have specific pathologies, including strategic misrepresentation (the polite term for lying to get projects approved), that don’t translate cleanly to typical software teams. A SaaS product built by a startup is not an airport.

More importantly, Flyvbjerg’s 0.5% figure measures whether projects deliver their promised benefits, not whether they hit their estimates. A project can hit its estimates perfectly and still fail to deliver value because the wrong thing was built. Conversely, a project can overrun significantly and still be wildly successful. Amazon Web Services famously launched years late and over budget, but it would be absurd to call it a failure.

That said, the track record for estimation accuracy in software specifically is still poor. Average cost overruns of 189% and time overruns of 222% aren’t anomalies; they’re common. Something is systematically wrong. But importing the worst-case statistics from megaprojects and presenting them as universal software truth overstates the case.

McConnell adds another sobering observation: projects hardly ever remain stable regarding the assumptions that affect estimates. Staff departs. Priorities shift. Budgets shrink. Markets change. The estimate made at the start described a project that no longer exists by the middle.

Why your brain lies to you

In brief: Cognitive biases make accurate estimation genuinely hard. The planning fallacy is robust across cultures, personality types, and experience levels. Even knowing about it doesn’t prevent it.

In 1979, Daniel Kahneman and Amos Tversky identified what they called the planning fallacy: the tendency to underestimate time, costs, and risks while overestimating benefits. This isn’t a bug in human cognition; it’s a feature. And it’s remarkably robust. It appears for small tasks and massive infrastructure projects. It generalises across personality types and cultures. It affects individuals and groups equally. Most troublingly, even knowing about the bias doesn’t prevent it.

When you estimate a software project, your brain deploys an arsenal of cognitive shortcuts. Optimism bias makes you believe you’re less likely to experience problems than others. Anchoring bias means the first number mentioned warps all subsequent thinking. The availability heuristic causes you to weight recent, memorable experiences more heavily than historical data. Self-serving bias leads you to take credit when things go well but blame external factors when they don’t, preventing you from learning.

Kahneman distinguished between the “inside view” and the “outside view”. The inside view focuses on a project’s specific characteristics: the features, the technology, the team. This feels like the right approach, but it systematically produces overconfident estimates. The outside view instead treats your project as one instance in a class of similar efforts and asks how those similar efforts actually performed. Kahneman called the outside view “the single most important piece of advice regarding how to increase accuracy in forecasting”.

This insight was formalised as Reference Class Forecasting: rather than estimating your project based on its unique characteristics, identify a reference class of similar past projects and use their actual outcomes as your baseline. The technique won Kahneman the Nobel Prize in Economics and has been mandated by the UK Treasury for large public projects. Yet software teams rarely implement it systematically, instead relying on expert judgment that demonstrably produces optimistic bias.

Ron Jeffries makes a related point: teams typically estimate “at the moment of maximum ignorance,” when understanding of what needs to be built is lowest. The estimates that get locked in as commitments are precisely the ones made when the team knew least about the work.

The Cone of Uncertainty: theory meets reality

In brief: McConnell’s Cone describes how uncertainty narrows on well-managed projects. The caveat is important: it only works when teams already practice healthy project management.

Steve McConnell popularised the Cone of Uncertainty as a framework for understanding estimation accuracy. The concept is elegant: early in a project, uncertainty spans a wide range (estimates may be off by 4x in either direction). As work progresses and unknowns resolve, the cone narrows. Variability decreases from 4x to 2x to 1.25x as projects move through requirements, design, and implementation phases.

The Cone offers useful insights. It legitimises early imprecision: demanding precise estimates during initial scoping ignores the fundamental reality that key decisions haven’t yet been made. It justifies iterative replanning: as information accumulates, estimates can legitimately narrow. Organisations that lock in early estimates and refuse updates are fighting mathematical reality.

But the critical caveat undermines the Cone as prescription. McConnell himself emphasises that the Cone describes what happens on well-managed projects. He writes: “It’s easily possible to do worse.” Many teams don’t systematically attack their highest sources of variability early. They postpone difficult problems, creating false certainty that shatters late in the project.

If the framework only works when teams already practice healthy project management, it describes a symptom of good practice rather than providing a path toward it. Teams struggling with estimation dysfunction can’t simply “follow the Cone.” They must first solve the organisational problems that prevent uncertainty from narrowing naturally.

The story points trap

In brief: Story points fail not because relative estimation is flawed, but because organisations misuse them. Even the practice’s inventor now expresses regret.

When Agile emerged, story points and velocity were supposed to solve the estimation problem. Abstract away from time. Focus on relative sizing. Let the data tell you how much you can deliver.

In theory, elegant. In practice, often dysfunctional.

The moment velocity becomes a performance metric, teams start gaming it. Story point inflation becomes rampant. Different teams have different reference scales, making cross-team comparisons meaningless, yet organisations constantly try to compare them. When hitting velocity targets matters more than sustainable delivery, teams cut corners. Technical debt accumulates. And despite story points being designed to avoid time estimation, stakeholders want dates, so organisations create conversion factors that destroy whatever value story points might have provided.

The inventor’s second thoughts

Perhaps the most striking critique of story points comes from Ron Jeffries himself, one of the original Extreme Programming founders often credited with inventing the practice. In 2012, Jeffries wrote:

“A team that is focusing on velocity is not focusing on value. I wish I had never invented velocity, if in fact I did.”

This isn’t nostalgic regret about misuse. Jeffries argues that velocity focuses on “the short end of the lever,” optimising the cost side of the equation when what matters in Agile is steering by selecting what to do and what to defer. He observed teams being measured on estimate accuracy, with product owners reduced to “just following the plan” rather than genuinely prioritising value.

The creator’s disillusionment doesn’t invalidate story points for all contexts. But it suggests that even under ideal conditions, the practice may misdirect attention. When your estimation system’s architect concludes it optimises the wrong variable, perhaps the problem runs deeper than organisational dysfunction.

The broader pattern

This matters because the same critique applies to any metric, including the flow metrics and throughput measures that forecasting advocates prefer. Throughput can be gamed too. Cycle time can be manipulated by how you define “started” and “done”. If leaders misuse metrics, changing metrics won’t fix leadership.

The Agile Manifesto’s seventh principle states: “Working software is the primary measure of progress.” Not story points. Not velocity. Not throughput. Working software. Any metric is a proxy, and any proxy can be gamed.

Forecasting: changing the conversation

In brief: Forecasting is still estimation. The value isn’t ontological (it doesn’t make prediction possible). The value is behavioural (it changes how estimates are discussed).

Around 2011, practitioners like Vasco Duarte, Neil Killick, and Woody Zuill started questioning the estimation orthodoxy under the hashtag #NoEstimates. They drew a distinction between estimating (giving a general idea based on opinion) and forecasting (calculating predictions based on historical data).

This distinction is useful, but we should be clear about what it actually changes. Forecasting is still estimation. When you run a Monte Carlo simulation and say “75% chance of completion by October 22nd”, you’re still making a prediction about the future. You’ve wrapped that prediction in probability language, making uncertainty explicit rather than hidden. That’s genuinely valuable, but it’s not magic.

The real value of probabilistic forecasting isn’t that it makes prediction possible where it wasn’t before. It’s that it changes the conversation. It’s harder to treat a probability distribution as a commitment. It forces discussions about risk tolerance. It makes uncertainty visible and therefore discussable.

Monte Carlo simulation, invented by John von Neumann and Stanislaw Ulam during World War II, uses repeated random sampling to model uncertain outcomes. For software delivery, you gather historical throughput data (how many items your team completes per week) and cycle time data (how long items take from start to finish). You then run thousands of simulations, randomly sampling from past performance to project possible futures. Instead of a single date, you get a probability distribution: 50% chance by October 15, 75% by October 22, 90% by November 1.

Daniel Vacanti puts it well: “If you want to get stuff done by August 31st, sure, we can get stuff done by August 31st, but there’s about a 40% chance of that happening. Are you okay taking that 60% risk? Now we can have a smarter, more adult economic conversation about what risk we’re willing to take.”

What honest estimation looks like

In brief: A concrete example of the difference between dysfunctional and healthy estimation conversations.

The difference between bad and good estimation isn’t the technique. It’s the conversation. Here’s what the same situation looks like handled badly versus handled well.

The dysfunctional version:

Product Manager: “When will the new payment integration be done?”

Tech Lead: “Hard to say exactly. There’s a lot of uncertainty around the third-party API.”

Product Manager: “I need a date for the roadmap.”

Tech Lead: “Um... maybe six weeks?”

Product Manager: “Great, I’ll put it down for March 15th.”

Four weeks later:

Product Manager: “We’re halfway through and you’re saying it might slip? You committed to March 15th.”

Tech Lead: “I said ‘maybe six weeks.’ And we’ve discovered the API doesn’t support batch operations, so we need to redesign.”

Product Manager: “This is going to be a difficult conversation with leadership. They’re expecting March 15th.”

The healthy version:

Product Manager: “When will the new payment integration be done?”

Tech Lead: “Based on similar integrations we’ve done, I’d say 50% chance we’re done in four weeks, 75% chance by six weeks, 90% by eight weeks. The big unknown is the third-party API. If it’s well-documented and supports our use cases, we’re at the fast end. If we hit surprises, we’re at the slow end.”

Product Manager: “Leadership wants it for the March launch. That’s six weeks away.”

Tech Lead: “So we’re looking at roughly 75% confidence for that date. What’s the cost if we miss it? Is there a fallback plan?”

Product Manager: “The launch can proceed without it, but it’s a headline feature. Missing it would be disappointing, not catastrophic.”

Tech Lead: “Then let’s proceed, but I’ll flag it early if we hit the API problems. If we’re in trouble by week three, we’ll know, and you’ll have time to adjust messaging.”

Four weeks later:

Tech Lead: “The API doesn’t support batch operations. We’re now tracking toward the six-to-eight week range. I wanted to let you know with two weeks still to go.”

Product Manager: “Thanks for the early warning. I’ll talk to leadership about plan B for the launch.”

The technique (Monte Carlo, throughput tracking, whatever) matters less than what’s happening in that conversation: ranges instead of points, explicit confidence levels, discussion of risk tolerance, early warning when forecasts change, and no blame when reality differs from prediction.

The assumptions behind forecasting

In brief: Forecasting requires stable throughput, comparable work items, and sufficient history. When these don’t hold, forecasts are unreliable, and you should say so.

Monte Carlo forecasting isn’t magic. It rests on assumptions that often go unstated.

It assumes reasonably stable throughput, meaning your team’s capacity tomorrow will resemble its capacity yesterday. It assumes comparable work items, meaning the things you’re forecasting are similar in nature to the things in your historical data. It assumes sufficient historical data to sample from. And it assumes no major structural changes to team composition, domain, or architecture.

These conditions often don’t hold. Greenfield products have no history. Early startups pivot constantly. Platform rewrites involve genuinely novel technical challenges. Regulated environments have external constraints that historical data can’t capture. Teams undergoing rapid scaling have unstable throughput by definition.

The situations where estimation is hardest are precisely the situations where you have the least relevant historical data to sample from. A team that’s been delivering similar features for two years can forecast with reasonable confidence. A team building something genuinely new, in a domain they don’t know well, with technology they haven’t used before, has no meaningful reference class.

For teams without historical data, the honest answer is: your forecasts will be unreliable, and you should say so. Start collecting data immediately, make small commitments, and let your forecasting improve as your history grows. Pretending you can forecast accurately without data isn’t forecasting; it’s just the old estimation game with fancier vocabulary.

Cognitive biases don’t disappear with data

In brief: Forecasting moves bias into different places (how work is sliced, which history is included) rather than eliminating it. Discipline still required.

The argument that estimation suffers from cognitive biases while forecasting doesn’t is too clean. The same biases that corrupt our estimates also affect how we collect and interpret historical data.

Forecasting moves bias into different places rather than eliminating it. Bias affects how work is sliced: teams often break down work in ways that make historical comparisons favourable. Bias affects which history is included: it’s tempting to exclude that disaster project as an “outlier” that doesn’t represent normal performance. Bias affects how outliers are treated: do you cap them, exclude them, or let them skew your distribution?

Teams tend to remember successes more vividly than failures. They categorise work items in ways that flatter past performance. The person choosing which historical items to include in the reference class brings all their biases with them.

This doesn’t mean data-driven forecasting is worthless. It just means it’s not the silver bullet it’s sometimes presented as. Good forecasting requires discipline: consistent categorisation, honest inclusion of failures, resistance to cherry-picking. The methodology helps, but it doesn’t eliminate the need for intellectual honesty.

The small stories alternative

In brief: If work items are small enough, estimation becomes almost unnecessary. You count items and measure throughput. The challenge lies in achieving that granularity.

Ron Jeffries argues that if work items are small enough, estimation becomes almost unnecessary. When everything takes roughly a day, you don’t need sophisticated forecasting. You count items and measure throughput. The challenge lies in achieving that granularity.

Technique 1: Single Acceptance Test Method

Neil Killick, cited by Jeffries, proposes examining acceptance criteria and implementing them one at a time, starting with the simplest. Rather than estimating “User can pay for order” as a single story, break it into separate work items:

Display payment button (simplest). Accept credit card number. Validate card format. Process test transaction. Handle payment failure gracefully. Store transaction record. Send receipt email.

Each acceptance test becomes a separate work item. Most will be small, genuinely completable in a day. The few that aren’t get broken down further.

Technique 2: The “One Dumb Idea” approach

Jeffries describes a psychological technique for unlocking small stories. When teams face seemingly monolithic features, propose a deliberately inadequate but technically possible first step.

His cable TV example: Instead of building full pay-per-view functionality, propose “Play one specific movie on a secret channel.” This exists almost entirely with current infrastructure. No user selection, no payment, no scheduling. Just hard-code a movie playing on channel 999.

The power lies in team psychology. When someone proposes an obviously insufficient solution, others instinctively respond with “What we could do instead is...” Suddenly the conversation shifts from “this is impossible” to discussing achievable increments. As Jeffries notes: “We’ve gone, in one step, from ‘impossible’ to knowing a stupid, but possible, thing to do.”

Technique 3: Vertical slicing with minimal viability

Each small story should deliver something end-to-end, however minimal.

Horizontal slicing (avoid): Build database schema. Create API endpoints. Implement frontend components. Wire everything together.

Vertical slicing (prefer): User can see one hardcoded product (end-to-end). User can see one product from database. User can see list of products. User can filter products by category.

The vertical approach produces shippable increments and reveals integration problems immediately rather than concentrating them at the end.

Why small stories help estimation

Even if you don’t eliminate estimation entirely, small stories transform the accuracy problem.

Reduced variability: A 10-day estimate might be off by 5 days. Ten 1-day estimates won’t all be wrong in the same direction.

Faster feedback: When items complete daily, you discover problems within days rather than weeks.

Throughput becomes measurable: With sufficient small items, historical throughput data emerges quickly, enabling forecasting without per-item estimation.

Cognitive load decreases: Estimating “this takes about a day” requires less analysis than forecasting two-week epics.

This approach requires investment in slicing skills and may initially feel slower than rougher-grained planning. But Jeffries argues the payoff is faster delivery with less estimation overhead, and crucially, less opportunity for estimates to become weaponised commitments.

The NoEstimates philosophy

In brief: The actual argument isn’t “don’t estimate” but “continuously question whether estimation is earning its keep.”

The NoEstimates movement, associated with Woody Zuill, Vasco Duarte, and Ron Jeffries, often gets reduced to “don’t estimate.” The actual argument is more nuanced: continuously question whether estimation is earning its keep.

Estimation as expense

Jeffries frames it starkly: “Estimates are always waste; they are not our product.” From a lean perspective, any activity that doesn’t directly produce customer value is expense. Estimation doesn’t ship features. The question becomes: does this expense generate sufficient return in decision quality to justify its cost?

On the C3 payroll project (Extreme Programming’s flagship case study), Jeffries’ team initially used story estimates for planning. They later realised they could have achieved similar outcomes by breaking work into single acceptance tests and counting completions. Mechanical measurement replacing estimation entirely.

When estimation provides value

Jeffries acknowledges legitimate use cases.

Sales and contracts: Pricing decisions require some basis. Though he critiques how organisations weaponise estimates in negotiations, he concedes that customers reasonably want cost projections before committing.

Understanding: Estimation discussions surface differing interpretations of requirements. Team members discover they imagined different solutions. However, Jeffries suggests the conversation provides this value. Written estimates aren’t strictly necessary.

Learning: Comparing estimates to actuals reveals systematic biases and process problems. Yet alternative monitoring methods exist; you can track cycle time without estimating individual items.

The pragmatic position

Jeffries’ conclusion is measured: “We always could stop estimating, but it’s not always the right thing to do. It’s always legitimate to think about it.”

This isn’t dogma. It’s a heuristic for continuous improvement. Each time estimation seems mandatory, ask: Is there a way to make decisions without this expense? What would we lose? What might we gain? Sometimes the answer favours estimation. Often, teams discover they estimate from habit rather than necessity.

When estimation is unavoidable

In brief: Fixed-price contracts, regulatory deadlines, capital budgeting, and external coordination all require estimates. The answer is to estimate honestly, not to pretend estimation is unnecessary.

Some contexts don’t allow the luxury of “we’ll deliver what we can when we can”. In these situations, estimation isn’t optional, and the question becomes how to do it less badly.

Fixed-price contracts require estimates. A client asking for a quote needs a number, and “it depends” doesn’t win business. You can build contingency into the price, you can structure contracts with change mechanisms, but you can’t avoid making a forward-looking commitment.

Regulatory commitments have hard deadlines. If compliance with a new regulation is required by a specific date, missing it has consequences that probabilistic language doesn’t soften. You need to know whether you’re likely to make it, and if not, what to do about it.

Capital budgeting requires forecasts. Organisations allocate resources annually or quarterly. Someone deciding whether to fund your initiative versus a competing one needs to understand what they’re getting for their investment. “Trust us” isn’t a capital allocation strategy.

External stakeholder negotiations depend on estimates. If you’re coordinating with partners, aligning marketing campaigns, or scheduling dependent work streams, those stakeholders need something to plan against.

In these situations, the answer isn’t to pretend estimation is unnecessary. It’s to estimate honestly: provide ranges rather than points, communicate confidence levels, update forecasts as you learn, and build relationships where changing estimates isn’t treated as failure.

Estimating less badly

In brief: Ranges, confidence levels, rolling-wave planning, Bayesian updating, and treating estimates as decision-support rather than promises.

When estimation is required, several practices help reduce the damage.

Use ranges instead of points. “Two to four weeks” is more honest than “three weeks” and gives stakeholders useful information about uncertainty. If they need the optimistic end, they know it’s a stretch. If they need certainty, they can plan for the pessimistic end.

Express confidence levels explicitly. P50, P75, and P90 estimates communicate that different levels of certainty come with different timelines. A P50 estimate means there’s a 50% chance of missing it. If that risk is unacceptable, plan for P90.

Practice rolling-wave planning. Estimate near-term work in detail, further-out work in ranges, and distant work as rough orders of magnitude. Don’t pretend you know what you’ll discover.

Update estimates as you learn. Bayesian updating means revising your forecasts as new information emerges. An estimate made at the start of a project should evolve. Treating the original estimate as a commitment regardless of what you’ve learned is organisational dysfunction, not estimation failure.

Treat estimates as decision-support, not promises. The purpose of an estimate is to help someone make a decision: should we fund this, should we commit to this date, should we staff this team. Once the decision is made, the estimate has served its purpose. Holding people to it regardless of changed circumstances misunderstands what estimates are for.

McConnell uses a helpful metaphor: estimates need not be perfect, just close enough that minor adjustments (equivalent to “sitting on the suitcase”) achieve reasonable success. Obsessing over estimation precision often misses the point.

Hybrid approaches

In brief: Use rough estimates early and forecasts later. Combine discovery and delivery. Layer probabilistic methods on top of expert judgment.

The debate is often framed as estimation versus forecasting, as if you must choose one camp. In practice, hybrid approaches often work best.

Use rough estimates early, forecasts later. At the inception of a project, you lack historical data for the specific work. Rough expert estimates, honestly communicated as guesses, help with initial go/no-go decisions. As work progresses and you accumulate data, shift to probabilistic forecasting based on actual throughput.

Combine discovery and delivery tracks. Run a time-boxed discovery phase to reduce uncertainty before committing to estimates. The goal of discovery is to learn enough that your subsequent estimates have a meaningful basis. Don’t estimate what you haven’t explored.

Use scenario-based planning. Instead of a single estimate, develop scenarios: “If the integration goes smoothly, four weeks. If we hit the authentication complexity we suspect, eight weeks. If we need to rebuild the data layer, three months.” This surfaces the key risks and lets stakeholders understand what drives the uncertainty.

Layer probabilistic forecasts on top of rough scoping. Use expert judgment to identify the likely scope, then apply Monte Carlo to the execution. The forecast doesn’t replace judgment about what needs to be built; it provides rigour around how long building takes.

McConnell advocates something similar: define requirements upfront with enough detail for story point estimation, then track velocity to calibrate forecasts. This offers a middle path between pure NoEstimates and traditional detailed estimation.

The economics of estimation

In brief: Cost of delay, risk-adjusted ROI, opportunity cost, and option value. If we want “adult economic conversations”, we should actually talk economics.

One thing largely missing from the estimation debate is economic decision theory. If we want “adult economic conversations”, we should actually talk economics.

Cost of delay matters enormously. A feature delivered in January might be worth twice what it’s worth in June. If you’re choosing between a certain six-month delivery and a risky four-month delivery, the right choice depends on how value decays over time. Probabilistic forecasting is most useful when connected to explicit cost-of-delay analysis.

Risk-adjusted return on investment changes decisions. A project with an expected value of £1 million but high variance might be less attractive than one with an expected value of £800,000 and low variance. Portfolio thinking requires understanding not just expected outcomes but distributions of outcomes.

Opportunity cost is invisible but real. While your team spends six months on Project A, they’re not working on Projects B, C, and D. The value of better estimation isn’t just delivering A faster; it’s making better choices about whether to do A at all.

Option value exists in uncertainty. Sometimes the right response to uncertainty isn’t better estimation; it’s structuring work to preserve options. Small investments that let you learn before committing are often worth more than precise forecasts that lock you in.

Fixed time, variable scope

In brief: Shape Up’s approach works well for product-led organisations with autonomy. It doesn’t work for regulatory deadlines or fixed external commitments.

Basecamp’s Shape Up methodology offers an interesting flip. Instead of fixing scope and letting time vary, you fix time and let scope vary. You decide that something is worth six weeks of effort, then build the best version you can in six weeks.

This is liberating in some contexts. Instead of expanding timelines to fit scope (which leads to Parkinson’s Law), you ruthlessly trim scope to fit timelines. Six weeks turns out to be a sweet spot: long enough to finish something meaningful, short enough to see the end from the beginning.

But Shape Up is context-specific, not universally applicable. It works well for product-led organisations with strong product management, high team autonomy, and low external deadline pressure. It works poorly for contract-based delivery, regulatory milestones, hardware dependencies, and systems with heavy multi-team coordination.

If the business requirement is “we need these specific regulatory features by this compliance deadline”, you can’t negotiate scope. If you’re coordinating with external partners who expect specific functionality, you can’t just deliver “whatever fits”. Shape Up is a valuable tool where it applies, not a universal solution.

The selection bias in success stories

In brief: Teams that work without traditional estimates have usually earned that trust through years of reliable delivery. Many teams don’t have that luxury.

Teams that successfully operate without traditional estimates share characteristics that often go unmentioned. They typically have high trust with stakeholders built over years of reliable delivery. They have mature development practices: continuous integration, automated testing, small incremental releases. They have stable funding that doesn’t require competitive justification.

In other words, they’ve earned the right to say “trust us”. They can operate with probabilistic forecasts because their stakeholders have seen enough delivery to believe the probabilities are meaningful.

Many teams operate in environments where that trust hasn’t been established. Funding is competitive. External dependencies require coordination. Stakeholders have been burned before and want commitments. Telling those teams to “just stop estimating” isn’t practical advice. They need to build trust first, which often means delivering reliably against stated expectations, which requires some form of estimation.

Flow metrics: the foundation

In brief: Cycle time, throughput, WIP, and work item age. These measure what actually happened rather than what someone guessed. But they can be gamed too.

Whatever approach you take, tracking the right metrics matters. The four essential flow metrics are cycle time (how long work takes from start to finish), throughput (how many items you complete per time period), work in progress (how many items are currently in flight), and work item age (how long an item has been in progress).

These connect through Little’s Law: average cycle time equals average work in progress divided by average throughput. Limiting WIP decreases cycle time. Decreased cycle time increases predictability. Increased predictability makes forecasting more accurate.

Teams that shift from story points and velocity to flow metrics often report reduced cycle times and better predictability. The mechanism is straightforward: flow metrics measure what actually happened, while story points measure what someone guessed would happen.

But remember: these metrics can be gamed too. The value comes from honest measurement and continuous improvement, not from the metrics themselves. Any metric that becomes a target ceases to be a good metric. The solution is cultural, not methodological.

The real enemy: organisational dysfunction

In brief: The problem isn’t estimation. It’s what organisations do with estimates. False precision is the enemy, not prediction itself.

Here’s the deeper issue the estimation debate often misses: the problem isn’t estimation itself. It’s the organisational dysfunction that surrounds it.

A team saying “two to four weeks, depending on what we discover” is estimating, and that’s fine. The problem is when that becomes “we committed to two weeks” in a status report, which becomes “why did you miss your commitment?” in a performance review. The dysfunction isn’t the estimate; it’s what the organisation does with it.

False precision is the enemy, not estimation. When someone asks “how long will this take?” and the honest answer is “probably 2-4 weeks, but it could be longer if we hit complications”, the organisation needs to be able to hear that. If it can’t, if it demands a single number and then treats that number as a commitment, the problem is cultural, not methodological.

The responsibility question

Ron Jeffries raises a pointed question about accountability. Developers cannot reasonably be held responsible for meeting deadlines without corresponding authority. Unless developers can hire or reassign staff, acquire additional resources, decide which features ship versus defer, or adjust scope unilaterally, they cannot control the variables that determine delivery dates.

Jeffries compares development to a machine with fixed capacity. Pushing harder doesn’t increase throughput; it risks breakdown. The product owner must select work batches that fit the timeline, not demand the machine work faster.

This has uncomfortable implications. When projects fail to meet deadlines, the conventional response is blaming developers for poor estimates. Jeffries suggests the failure often lies in management’s scope decisions, resource allocation, or unrealistic targets, factors developers cannot control.

What developers can commit to: delivering working, tested features regularly; keeping the codebase shippable at all times; working on whatever sequence management prioritises; surfacing impediments early rather than hiding them. This is narrower accountability than “hit the date,” but it’s accountability developers can actually fulfil without authority over scope and resources.

Dysfunction arises when organisations hold people accountable for outcomes they cannot control. Estimates become the instrument for manufacturing this false accountability.

The cultural barrier

Monte Carlo simulations and probabilistic forecasting help with the honesty part. They make it harder to pretend certainty where none exists. But they don’t solve the cultural part. An organisation that punishes missed estimates won’t suddenly become healthy because you started expressing estimates as probability distributions. They’ll just punish you for missing the 75th percentile instead of the point estimate.

What actually helps

In brief: Track reality, embrace probability, keep things small, focus on flow, estimate honestly when you must, and build a culture that can handle uncertainty.

Based on all the evidence, here’s what actually improves outcomes.

Track what actually happens. Historical throughput beats expert guesses. Collect cycle time and throughput data consistently, even if you’re not sure how you’ll use it yet.

Embrace probabilistic thinking. Present ranges with confidence levels rather than single dates. Have risk conversations rather than commitment ceremonies. Acknowledge that you’re uncertain and explain what drives the uncertainty.

Keep work items genuinely small. Target 1-day items where possible, certainly no more than 3 days. Use the slicing techniques: single acceptance tests, vertical slicing, the “one dumb idea” approach. Smaller items mean tighter distributions and better forecasts.

Focus on flow. Measure cycle time and throughput. Limit work in progress. The maths is clear: lower WIP means faster cycle times and more predictable delivery.

Where possible, fix time and vary scope. If you can negotiate what gets built, time-boxing forces real prioritisation and reduces the scope creep that derails traditional projects.

When estimation is required, do it honestly. Use ranges, express confidence levels, update as you learn, and treat estimates as decision-support rather than commitments. Separate estimates from targets from commitments in your vocabulary and your conversations.

Build organisational tolerance for uncertainty. This is the hardest part and the most important. If your organisation punishes honest uncertainty, no methodology will save you. Work on the culture alongside the practices.

What about AI?

In brief: AI agents change the nature of the work being estimated. Historical data becomes less relevant. Variance increases. The estimation problem gets harder before it gets easier.

AI coding agents are already changing how software gets built. Tasks that took a day now take an hour. But not all tasks, and not predictably. This creates a new estimation problem: how do you forecast when the work itself is transforming?

Historical throughput data assumes some stability in how work gets done. If your team’s cycle time for “build a new API endpoint” was consistently 2-3 days, you could forecast based on that. But now one developer with an AI agent finishes it in two hours, while another developer working on a different endpoint hits edge cases the AI can’t handle and takes three days anyway. Your historical distribution no longer describes your current capability.

The variance problem gets worse, not better. AI agents are fast when they work and useless when they don’t, and predicting which situation you’ll hit is difficult. A task might complete in minutes if the AI handles it cleanly, or take longer than the pre-AI baseline if you spend hours debugging AI-generated code that almost works. The distribution of outcomes becomes bimodal or worse, which breaks the assumptions behind Monte Carlo forecasting.

There’s also a decomposition problem. Traditional estimation assumes humans do the work and you’re estimating human effort. When AI agents do significant portions of the work, what exactly are you estimating? The human time spent prompting, reviewing, and correcting? The wall-clock time including AI processing? The cognitive load on the human, which might be higher when supervising AI than when doing the work directly? None of our existing frameworks handle this cleanly.

The honest answer is that we don’t yet know how to estimate human-AI collaborative work well. Teams adopting AI agents should expect their forecasting accuracy to degrade temporarily. Historical data becomes less useful. New patterns haven’t stabilised enough to replace the old ones. The best approach is probably radical incrementalism: even smaller batches, even shorter feedback loops, even more willingness to update forecasts as you learn. Treat every AI-assisted task as an experiment until you’ve built enough new history to see patterns.

This might ultimately be good news for the estimation debate. If AI makes historical data unreliable and variance unpredictable, organisations will be forced to accept uncertainty whether they like it or not. You can’t demand false precision when everyone can see the ground shifting. But in the short term, expect estimation to get harder, not easier.

Conclusion: three voices, one uncomfortable truth

We’ve heard three distinct perspectives on software estimation.

Steve McConnell argues that estimation is a craft that can be practiced skilfully. Distinguish estimates from targets from commitments. Understand the Cone of Uncertainty. Use historical data and reference classes. Track actuals. The problem isn’t estimation itself but doing it carelessly.

Ron Jeffries questions whether estimation deserves its central role. Every estimate is expense, not product. When work is sliced small enough and delivered continuously, forecasting emerges from counting rather than guessing. At minimum, keep asking: can we make this decision without estimating?

The organisational dysfunction thesis identifies cultural problems as the root cause: not estimation technique or philosophy, but what organisations do with estimates, demanding false precision, punishing honest uncertainty, treating preliminary forecasts as ironclad commitments.

These perspectives aren’t as contradictory as they first appear.

McConnell would agree that estimation without distinguishing it from commitment is organisational malpractice. Jeffries would agree that when estimation is necessary, doing it well beats doing it badly. All parties agree that probabilistic language beats point estimates, that smaller work items improve predictability, and that organisational culture determines whether any technique succeeds.

McConnell offers: “The primary purpose of software estimation is not to predict a project’s outcome; it is to determine whether a project’s targets are realistic enough to allow the project to be controlled to meet them.”

Jeffries offers: “Stop estimating. Start shipping.”

Perhaps both are right. When an organisation has the maturity to use estimates as McConnell envisions, for project control rather than prophecy, estimation becomes a valuable tool. When an organisation lacks that maturity, Jeffries’ provocative advice may be the safest path: stop the estimation theatre entirely, focus on small deliverables, and let observable throughput speak for itself.

The question isn’t “Should we estimate?” but “Has our organisation earned the right to estimate responsibly?”

That requires cultural change harder than switching methodologies, but it’s the only approach that genuinely works.

References

Steve McConnell

McConnell, S. (2006). Software Estimation: Demystifying the Black Art. Microsoft Press.

McConnell, S. “Software Engineering Radio Episode 273: Steve McConnell on Software Estimation.” Software Engineering Radio.

McConnell, S. “The Cone of Uncertainty.” Construx. https://www.construx.com/books/the-cone-of-uncertainty/

Ron Jeffries

Jeffries, R. “The NoEstimates Movement.” ronjeffries.com. https://ronjeffries.com/xprog/articles/the-noestimates-movement/

Jeffries, R. “Estimation is Evil.” ronjeffries.com. https://ronjeffries.com/articles/019-01ff/estimation-is-evil/

Jeffries, R. “Getting Small Stories.” ronjeffries.com. https://ronjeffries.com/articles/015-10/small-stories/

Jeffries, R. “Making the Date.” ronjeffries.com. https://ronjeffries.com/articles/making-the-date/

Jeffries, R. “Story Points Revisited.” ronjeffries.com. https://ronjeffries.com/articles/019-01ff/story-points/Index.html

Jeffries, R. (2012). Comment on Scrum Alliance discussion regarding velocity. Referenced in InfoQ: “Should we stop using Story Points and Velocity?”

Daniel Kahneman and Amos Tversky

Kahneman, D. & Tversky, A. (1979). “Intuitive Prediction: Biases and Corrective Procedures.” TIMS Studies in Management Science, 12, 313-327.

Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux.

Reference Class Forecasting

Flyvbjerg, B. (2006). “From Nobel Prize to Project Management: Getting Risks Right.” Project Management Journal, 37(3), 5-15.

Wikipedia. “Reference Class Forecasting.” https://en.wikipedia.org/wiki/Reference_class_forecasting

Project Statistics

Flyvbjerg, B. (2021). “How Big Things Get Done.” Oxford Saïd Business School.

The Standish Group. “CHAOS Report.” https://www.standishgroup.com/

NoEstimates Movement

Duarte, V. NoEstimates: How to Measure Project Progress Without Estimating. Leanpub. https://leanpub.com/noestimates

Zuill, W. “NoEstimates.” https://zuill.us/WosenseeBlog/tag/noestimates/

NoEstimates.org. Links and Resources. https://www.noestimates.org/

Flow Metrics and Forecasting

Vacanti, D. When Will It Be Done? Lean-Agile Forecasting to Answer Your Customers’ Most Important Question. Leanpub. https://leanpub.com/whenwillitbedone

Vacanti, D. Actionable Agile Metrics for Predictability. Leanpub. https://leanpub.com/actionableagilemetrics

Kersten, M. (2018). Project to Product: How to Survive and Thrive in the Age of Digital Disruption with the Flow Framework. IT Revolution Press.

DORA Research

Forsgren, N., Humble, J., & Kim, G. (2018). Accelerate: The Science of Lean Software and DevOps. IT Revolution Press.

DORA. “DORA Metrics: The Four Keys.” https://dora.dev/guides/dora-metrics-four-keys/

Shape Up

Singer, R. Shape Up: Stop Running in Circles and Ship Work that Matters. Basecamp. https://basecamp.com/shapeup

Additional Sources

Killick, N. “Slicing Heuristics.” Referenced in Jeffries’ work on small stories.

Beck, K. et al. “Manifesto for Agile Software Development.” https://agilemanifesto.org/

“Story points aren’t about the numbers. They trigger conversations about effort and complexity.”

I hear this defence constantly. It sounds reasonable on the surface. Almost wise. But I want you to think carefully about what it’s actually saying, because underneath the reasonable-sounding words lies a significant admission.

If the real value of story points is the conversation they trigger, why do you need story points to have that conversation?

You could just talk about the work. Directly. Without the numerical theatre. Without the planning poker cards. Without the endless debates about whether this piece of work is a 3 or a 5 or maybe an 8.

The fact that we need to dress up “having a conversation about work” in estimation clothing should make us suspicious. It suggests that something else is going on. And that something else is where the danger lies.

Let me be clear about what I mean when I say story points are dangerous. I don’t mean they’re mildly unhelpful or slightly inefficient. I mean they actively harm teams, damage trust, and create dysfunction. They do this in ways that are subtle enough to go unnoticed for years, which makes them more dangerous, not less.

Story points don’t measure anything real

When you measure something in metres, everyone agrees what a metre is. When you measure something in kilograms, we have a shared understanding of the unit. Story points don’t work this way. A “point” means something different to every team, and often means something different to every person on the same team.

Some people think about points as effort. How much work will this take? Some people think about points as complexity. How many moving parts are involved? Some people think about points as risk. How likely is this to go wrong? Some people think about points as uncertainty. How much do we not know?

These are all different things. Effort and complexity are related but not the same. A task can be simple but effortful, like copying a thousand records by hand. A task can be complex but quick, like making a small change to a critical algorithm where you need to understand the whole system but the actual change is tiny.

When your team sits down to estimate, each person is measuring a different thing with the same number. You’re using the same scale to weigh apples, measure distances, and count sheep. Then you’re surprised when the numbers don’t mean anything useful.

Agreement on a number creates an illusion of shared understanding

This problem is more insidious than the first. Imagine your team is estimating a story. After some discussion, everyone holds up a 5. Success, right? You’ve reached consensus. The process worked.

But did it?

Person A is thinking about the database changes required. They’ve estimated a 5 because they know the schema is messy and migrations are risky.

Person B is thinking about the front-end work. They’ve estimated a 5 because there are several screens to update and the design isn’t finalised.

Person C is thinking about the integration with the external API. They’ve estimated a 5 because they’ve never worked with this API before and the documentation looks sparse.

Person D is thinking about testing. They’ve estimated a 5 because this feature touches several existing workflows and regression testing will be extensive.

Everyone said 5. Everyone had completely different reasons. They’re not estimating the same work. They’re not even thinking about the same aspects of the work.

The number created false consensus. It let everyone believe they were aligned when they weren’t. And this false consensus is worse than open disagreement, because open disagreement gets resolved. False consensus just waits quietly until it explodes during implementation.

If Person A starts working on this story, they might ignore the front-end complexity that Person B was worried about. If Person C picks it up, they might not realise the database concerns that Person A had in mind. The “shared estimate” shared nothing except a digit.

This is what I mean when I say story points are dangerous. They create a feeling of alignment without actual alignment. They let teams believe they’ve communicated when they haven’t. They substitute a ritual for real understanding.

The retreat to “valuable conversations”

Now let’s talk about what happens when teams try to defend story points.

The most common defence is the one I started with: story points trigger valuable conversations. But notice what this defence concedes. It concedes that the points themselves aren’t valuable. It concedes that accuracy doesn’t matter. It concedes that the numbers are essentially arbitrary.

If someone told you that their weighing scale was completely inaccurate but it was still valuable because stepping on it reminded them to think about their health, you’d suggest they just think about their health directly and throw away the broken scale.

The “triggers conversations” defence is what people say when they can no longer defend the thing on its own merits. It’s a retreat to secondary benefits. And those secondary benefits can be achieved more directly without the harmful primary effects.

If you want to understand complexity, ask “what makes this complicated?” directly.

If you want to surface risks, ask “what could go wrong?” directly.

If you want to check whether the team has shared understanding, ask “how would we approach this?” directly.

If you want to know if everyone is thinking about the same work, ask “what do you think this story involves?” directly.

These questions actually achieve the supposed goals of estimation. They surface different perspectives. They reveal hidden assumptions. They create genuine shared understanding rather than the illusion of it.

And crucially, they don’t produce a number that will later be misused.

Story points get weaponised

Because here’s the third danger of story points: they get weaponised.

No matter how many times you tell management that story points aren’t comparable across teams, they will compare them across teams. No matter how many times you explain that velocity isn’t a productivity measure, it will be used as a productivity measure. No matter how many times you insist that points shouldn’t be converted to hours, someone will create a conversion formula.

Story points create numbers. Organisations love numbers. Numbers go into spreadsheets. Spreadsheets go into reports. Reports go to executives. Executives make decisions based on those numbers.

And now your meaningless, arbitrary, inconsistent, incomparable numbers are driving organisational decisions. Teams get pressured to increase velocity. Developers learn to inflate estimates so they can “deliver more points.” Gaming begins. Trust erodes.

You might say this is a misuse of story points, not a problem with story points themselves. But this misuse is inevitable. It happens in nearly every organisation that adopts story points. If a tool is consistently misused across thousands of different organisations with different cultures and different people, the problem is the tool, not the users.

A good tool makes the right thing easy and the wrong thing hard. Story points make the wrong thing easy. They produce numbers that look meaningful, invite comparison, and beg to be aggregated. They’re practically designed to be misused.

The myth of improving at estimation

Let’s look at another defence: story points help teams improve at estimation over time.

This sounds plausible. Practice makes perfect, right? But there’s a hidden assumption here that deserves examination. The assumption is that there’s a stable, learnable skill called “estimation” that teams can get better at.

In reality, every piece of work is different. The factors that made your last story take longer than expected are probably not the factors that will affect your next story. Software development is not like manufacturing widgets, where you can measure cycle time and optimise a repeatable process. Each story involves different code, different requirements, different unknowns.

Teams that track their estimation accuracy over time often find that it doesn’t improve. Or it improves for a while and then degrades. Or it varies randomly with no clear trend. This isn’t because the teams are bad at learning. It’s because there isn’t a stable underlying skill to learn.

What teams actually get better at is playing the estimation game. They learn what numbers the organisation wants to hear. They learn to pad estimates to create safety margin. They learn to break work into smaller pieces not because smaller pieces are better but because smaller estimates are less scrutinised.

The improvement is an illusion. Or worse, the improvement is real but it’s improvement at gaming the system rather than improvement at understanding work.

The false promise of avoiding false precision

Here’s another defence I hear: story points are better than time-based estimates because they avoid the trap of false precision.

This defence has some truth in it. Time-based estimates do create false precision. Saying “this will take 3.5 days” implies a level of accuracy that doesn’t exist.

But story points don’t solve this problem. They just move it. Instead of false precision about time, you get false precision about relative size. Saying “this is a 5” implies that you know how this work compares to other work, that you understand the scope well enough to place it on a scale, that the number means something.

And then the false precision comes back anyway, because most organisations convert velocity into time-based forecasts. If you complete 30 points per sprint and your backlog has 300 points, you forecast 10 sprints. The time-based precision you tried to avoid has snuck back in through the back door.

You haven’t eliminated false precision. You’ve just added a layer of indirection that makes the false precision harder to see and challenge.

What story points do to how teams think

Now I want to address the deepest problem with story points, which is what they do to the way teams think about work.

Story points encourage teams to think about work as a quantity to be estimated rather than a problem to be understood.

When you approach work through the lens of estimation, you’re asking “how big is this?” That’s a question about the work as a fixed object, something with a predetermined size that you’re trying to measure.

But software development doesn’t work that way. The work isn’t fixed. The scope isn’t predetermined. The size emerges from how you approach the problem, what trade-offs you make, what you discover along the way.

When teams approach work through the lens of understanding, they ask different questions. What are we trying to achieve? What’s the simplest thing that could work? What do we need to learn? What could we defer?

These questions lead to better outcomes because they engage with the work as something to be shaped rather than something to be measured. They open up possibilities rather than closing them down into a single number.

I’ve watched teams spend an hour debating whether a story is a 5 or an 8. That’s an hour they could have spent actually understanding the work. Actually talking about the approach. Actually identifying risks and unknowns. Actually building shared mental models.

The estimation ritual crowds out the valuable activities. It substitutes a proxy for the real thing. And because the proxy feels productive, feels like work, feels like alignment, teams don’t notice what they’re missing.

What happens when teams stop estimating

Let me talk about what happens when teams stop using story points.

The first thing that happens is fear. Managers worry about losing visibility. They ask how they’ll know when things will be done. They ask how they’ll track progress. They ask how they’ll compare teams.

These fears are worth examining. What visibility did story points actually provide? If the numbers were arbitrary and inconsistent and gameable, what were managers really seeing? They were seeing a theatrical performance of estimation, not a window into reality.

When teams stop estimating and start focusing on flow, something interesting happens. They start measuring things that actually matter. How long do items spend in progress? Where do items get stuck? How often do items get blocked? What’s the cycle time from start to finish?

These measurements are grounded in reality. They’re not opinions or guesses. They’re observations of what actually happened. And they’re much harder to game because they’re based on timestamps, not feelings.

Teams that focus on flow also tend to break work into smaller pieces, not because smaller estimates are safer, but because smaller pieces flow better. They hit problems earlier. They learn faster. They deliver value sooner.

This is a genuine improvement in the way teams work, not an improvement in the way teams estimate. And it happens precisely because teams stopped spending energy on estimation and redirected that energy toward actually improving their work.

What to do instead

I’m not saying all estimation is worthless. Sometimes you need to make decisions that require rough forecasts. Should we start this project? Can we deliver by this deadline? How should we staff this team?

But these decisions don’t require story points. They require honest conversations about uncertainty. They require looking at historical data. They require acknowledging what you don’t know.

A team that says “based on how similar work has gone in the past, this might take two to four weeks, but there are significant unknowns around the external integration” is providing more useful information than a team that says “we estimated this at 23 points and our velocity is 8 points per sprint.”

The first statement is honest about uncertainty. The second creates false precision that will be treated as a commitment.

Have the conversations you actually want to have

Let me return to where I started. “Story points trigger conversations about effort and complexity.”

If you find yourself defending a practice by pointing to its side effects rather than its primary purpose, it’s worth asking whether you should replace that practice with something that achieves the side effects directly.

If you want conversations about effort, have conversations about effort.

If you want conversations about complexity, have conversations about complexity.

If you want shared understanding, build shared understanding through discussion, collaboration, and working together on the actual work.

You don’t need a numerical ritual to have good conversations. The ritual gets in the way more often than it helps. The numbers create more problems than they solve.

Stop estimating. Start understanding. Your team will be better for it.

This is a non-sequitur.

Let’s start with an observation that should be obvious but apparently isn’t. Humans are non-deterministic too. Give the same problem to two engineers and you get two different solutions. Give it to the same engineer six months later and you still get a different one. That has never stopped us from shipping software.

The implicit comparison being made is between LLMs and compilers. Compilers are deterministic: same input, same output, every time. LLMs are not. Therefore LLMs are unreliable. But this comparison misses the point entirely. We don’t compare developers to compilers. We don’t expect humans to produce identical output given identical input. We expect them to produce correct output, verified through review and testing.

The same standard applies to AI tools.

Now, I’m not claiming LLMs fail the same way humans do. They don’t. LLMs hallucinate with confidence. They don’t self-correct the way a human rereads their own code and spots the mistake. They can produce plausible-looking nonsense that passes a casual glance. These are real failure modes, and they’re different from the failure modes we’re used to managing.

But this is precisely why we verify. This is why we test. This is why we review outputs rather than shipping them blindly. The failure modes are different, so our verification strategies need to account for them. That’s an engineering problem with engineering solutions.

What actually matters is behaviour, not how we got there. We instruct agents, review the output, and rely on tests as executable statements of expected behaviour. As long as the external behaviour is preserved, the internal path taken to reach it is largely irrelevant. This is exactly why test suites exist and why refactoring is even possible.

But there’s a deeper confusion at play here, one about agency and control.

When people say “AI is non-deterministic”, they often mean something more like “AI does unpredictable things on its own”. As if we were talking about a colleague who happens to be made of silicon. We’re not. We’re talking about software that responds to inputs.

I’ll grant that this can feel like an understatement when you’re watching an agent loop operate over time, calling tools, maintaining context, producing outputs that seem to emerge rather than follow directly from a prompt. There is real complexity here, and reasoning about agent behaviour does add cognitive load. I’m not dismissing that.

But complexity is not the same as autonomy. These systems don’t have intentions, preferences, or agency. They have behaviours that emerge from inputs, context, and training. The complexity makes them harder to reason about, not impossible to control.

And here’s what the non-determinism critics often miss: we have extensive control over what these tools do. Modern AI coding tools offer orchestration capabilities, subagents for specific tasks, and guardrails that constrain behaviour. You can define workflows, set boundaries, require confirmations. The tool doesn’t run wild. You direct it.

This is no different from how we’ve always worked with complex systems. We don’t trust developers to never make mistakes. We build processes around them: social programming, code review, automated testing, continuous integration, staged deployments. The system as a whole produces reliable outcomes even though individual contributors are thoroughly non-deterministic.

The question isn’t whether an LLM will give you the same answer twice. The question is whether you have the discipline to verify outputs, the tests to catch regressions, and the workflows to maintain control.

Here’s where I need to be direct about something. This argument assumes a certain level of engineering maturity. Tests, reviews, guardrails, orchestration, discipline. Critics might respond that many teams don’t have this maturity, and that AI makes their problems worse.

They’re not wrong.

AI is an amplifier. It amplifies whatever practices you already have. If you have strong testing habits, AI helps you move faster with confidence. If you ship code without verification, AI helps you ship bad code faster. This isn’t a reason to avoid AI tools. It’s a recognition that using them well requires the same foundations that any reliable software development requires. The bar hasn’t changed. The speed has.

This matters because language shapes thinking. If we talk about AI as if it has agency (yes, “agents” might be misleading), we start believing we can delegate not just tasks, but responsibility. We start thinking accountability can be outsourced to a machine.

It can’t. And this is where the human analogy reaches its limit. Humans aren’t just another non-deterministic process in the pipeline. We have intent, situational awareness, and moral judgement. AI tools have none of these. That’s precisely why responsibility remains with us. We’re not accountable because we’re in the loop. We’re accountable because we’re the only ones who can be.

We chose to use the tool. We chose to accept the output. We chose to ship it, publish it, send it. The outcome is ours, good or bad.

So when I hear the non-determinism argument, it sounds less like a real concern and more like a category error. We’re comparing the variability of a generative process to the reliability of a deterministic one, and concluding that the former is inherently unsuitable for serious work.

But we’ve been doing serious work with non-deterministic processes forever. They’re called humans.

AI doesn’t do anything. We do things with AI. That’s not pedantry. That’s the difference between using a tool responsibly and hiding behind one.

Both positions miss the point entirely. But the second one is particularly insidious because it sounds reasonable while defending against an attack that nobody serious is actually mounting.

When someone proclaims that “AI is not going to replace engineers,” they are constructing and demolishing a straw man. The interesting question was never about wholesale replacement. It never has been, for any technology, in any era. The real questions are far more uncomfortable, which is precisely why we avoid them.

Let me explain what I mean.

The replacement fantasy has ancient roots. When John McCarthy, Marvin Minsky, and their colleagues gathered at Dartmouth College in the summer of 1956, they proposed a study based on the conjecture that “every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it.” They estimated the project would take about two months with ten researchers.

This spectacular miscalculation set the template for AI discourse that persists to this day. The field has oscillated between manic overconfidence and depressive retrenchment ever since. The “AI winters” of the 1970s and late 1980s followed periods of grandiose promises that could not be kept. Expert systems were going to revolutionise everything until they didn’t. Neural networks were abandoned as dead ends until they weren’t.

Each cycle produces the same rhetorical pattern. Enthusiasts make sweeping claims about imminent transformation. Sceptics point to obvious limitations. The technology fails to deliver on the hype. Everyone declares AI overblown. Then, quietly, the technology improves, finds its niche, and reshapes things in ways nobody predicted.

The sceptics are always right in the short term and always wrong in the long term. The enthusiasts are always wrong in the short term and often right about directions if not timelines.

This brings us to the philosophical heart of the matter. The Hungarian-British polymath Michael Polanyi articulated what we now call Polanyi’s Paradox: “We can know more than we can tell.” Much of human expertise consists of tacit knowledge that we cannot easily articulate or formalise. Hubert Dreyfus, the philosopher who spent decades critiquing AI, built his case on similar foundations. He argued that human intelligence is fundamentally embodied, contextual, and resistant to the kind of rule-based formalisation that AI required.

These arguments were powerful. They were also, as it turned out, arguments against a particular approach to AI rather than against AI itself. The shift from symbolic AI to machine learning, from explicit programming to pattern recognition from data, sidestepped many of Dreyfus’s objections. He was right that you cannot write down rules for recognising faces or understanding natural language. He did not anticipate that you wouldn’t need to.

John Searle’s famous Chinese Room argument makes a different point. A person who follows rules to manipulate Chinese symbols, without understanding Chinese, does not thereby come to understand Chinese. The room processes the symbols correctly but comprehension is absent. Searle meant this as a refutation of strong AI claims about machine consciousness and understanding.

But here’s the uncomfortable truth: much of what we call software engineering does not require understanding in Searle’s deep sense. It requires symbol manipulation according to patterns. When an AI generates code that compiles, runs correctly, and solves the stated problem, whether it “understands” what it’s doing becomes philosophically interesting but practically irrelevant.

The defender of human engineers says: but what about the hard parts? What about architecture, requirements analysis, understanding the business domain? What about debugging subtle issues and making judgment calls under uncertainty?

These are fair points. They are also, almost certainly, temporary ones. The history of automation is the history of tasks that could never be automated until they were.

Consider the game of Go. For decades, it served as the exemplar of human intuition triumphing over brute computation. Chess might fall to Deep Blue, but Go was different. The branching factor was too large. The positional judgment was too subtle. Human intuition was irreplaceable. In 2016, AlphaGo defeated Lee Sedol, and that particular argument died.

The pattern repeats. We identify what makes current AI inadequate. We treat those limitations as fundamental. We build our professional identity on being the ones who can do what machines cannot. Then the machines learn to do it anyway.

This is where the straw man becomes dangerous. When engineers say “AI won’t replace us,” they are defending against total elimination. But total elimination is not what’s coming. What’s coming is something more like what happened to bank tellers, travel agents, and countless other professions that still exist but employ far fewer people at different tasks than before.

The economists have a term for this: labour-saving technology. When a technology allows one worker to produce what previously required five, you don’t need to replace all the workers. You just need fewer of them. The work still exists. Human judgment is still involved somewhere in the process. But four out of five workers are doing something else, if they’re lucky, or nothing at all, if they’re not.

The augmentation narrative that many engineers embrace is a comforting frame that obscures this dynamic. Yes, AI augments human capability. That’s precisely the problem. If I can now accomplish with AI assistance what previously required a team, the team is no longer required. The fact that a human remains in the loop is cold comfort to the humans removed from it.

Karl Marx, whatever one thinks of his prescriptions, was remarkably prescient about machinery and labour. He observed that machines do not simply make work easier. They transform the relationship between labour and capital. They create what he called a “reserve army” of unemployed workers whose presence disciplines those still employed. Technology under capitalism serves capital’s interests, not labour’s.

One need not be a Marxist to recognise the dynamic. When AI enthusiasts at technology companies promise that AI will augment developers, they are making a statement about capability. When they predict that this will not affect employment, they are making a statement about economics that does not follow from the first claim and that their own incentives make them unqualified to assess.

The philosophical tradition offers another useful concept here: the Ship of Theseus. If you replace every plank of a ship, one at a time, is it still the same ship? Apply this to software engineering. If AI takes over code generation, then testing, then documentation, then debugging, then architecture, each time with a human “in the loop” providing approval, at what point has the engineer been replaced while still technically being present?

The answer is that replacement is not a binary event but a gradient. You can be replaced by degrees. You can be diminished incrementally. You can find yourself nominally in control while actually serving as a rubber stamp for decisions made elsewhere. The straw man of total replacement distracts from this more likely and more insidious outcome.

Martin Heidegger wrote about technology as a mode of “revealing,” a way of disclosing the world that also conceals other possibilities. When we frame the question as “will AI replace engineers,” we reveal a binary choice and conceal the spectrum of possibilities in between. The framing itself does ideological work, making certain outcomes thinkable and others invisible.

What should we actually be asking? Not whether AI will replace engineers, but how AI will transform engineering, who will benefit from that transformation, what skills will become more or less valuable, and how we might influence these trajectories rather than simply reacting to them.

The Luddites, contrary to popular caricature, were not opposed to technology as such. They were skilled textile workers who objected to the use of machinery to circumvent labour standards and degrade working conditions. Their complaint was not that machines were bad but that machines were being used badly. History has vindicated the direction of their concerns while condemning the futility of their methods.

We might learn from them. The question is not whether to resist AI, which is as futile now as smashing looms was then. The question is whether to participate in shaping how AI transforms our profession or to console ourselves with the fairy tale that it won’t.

When someone tells you that AI won’t replace engineers, ask them what they mean. If they mean that engineers will not vanish overnight, they are trivially correct. If they mean that the number, nature, and compensation of engineering roles will remain unchanged, they are almost certainly wrong. If they mean that they personally will be fine, they may be right or may be engaging in the same wishful thinking that has consoled displaced workers throughout history.

The honest position is uncertainty. We don’t know exactly how this will unfold. The capabilities are advancing faster than most predictions. The integration into actual workflows is slower than the hype suggests. The economic incentives are clear but the timeline is not.

What we can say is that treating “AI won’t replace engineers” as a meaningful contribution to this conversation is an intellectual abdication. It’s a thought-terminating cliché that protects us from harder questions. It is the equivalent of standing on the shore, watching the tide come in, and reassuring ourselves that water cannot replace sand.

The sand will still be there when the tide recedes. It will just be in a very different place.

The phrase “don’t deploy on Friday” has become one of those pieces of engineering wisdom that people repeat without much examination. It sounds sensible. It feels responsible. And it is almost entirely wrong.

I recently read a post defending this rule. It was well-written and made several arguments that, on the surface, seem reasonable. But each of these arguments, when examined closely, reveals a deeper problem that the Friday rule does not solve but merely hides.

Let me address them one by one.

“You rarely have full control over everything involved in a deployment”

The argument goes like this: infrastructure, third-party services, configuration differences, traffic patterns, and environment-specific behaviour can all surface issues only after a change is actually applied. These are not always predictable in staging or test environments.

This is true. It is also completely irrelevant to the question of which day you deploy.

If your production environment behaves differently from staging in ways that cause incidents, that is a problem on Tuesday just as much as on Friday. The unpredictability does not magically disappear because you have five working days ahead of you. What changes is your ability to throw human hours at the problem.

The correct response to environmental unpredictability is not to limit your deployment days. It is to invest in observability, automated rollbacks, and production testing strategies like canary deployments and feature flags. These approaches reduce blast radius and enable fast recovery regardless of when something goes wrong.

If your only mitigation strategy for environmental surprises is “have people available to fix things manually”, you have not solved the problem. You have just accepted it.

“Data migrations are fundamentally different and risky”

The post argues that changes touching real data can be slow, irreversible, dependent on data volume and shape, and risky to roll back. Treating them the same as a small code deploy ignores real operational complexity.

I completely agree. Data migrations are different and require special handling.

But this is an argument for treating data migrations differently. It is not an argument for a blanket rule about which days any deployment can happen.

A well-designed migration strategy includes expand-and-contract patterns, backward-compatible schema changes, and the ability to run migrations independently from code deployments. If your migrations are so risky that they require days of potential cleanup time, the problem is not Friday. The problem is how you are doing migrations.

Conflating “don’t deploy risky migrations on Friday” with “don’t deploy anything on Friday” is sloppy thinking. It takes a legitimate concern about one specific type of change and inflates it into a universal policy that restricts all changes. This is how reasonable caution turns into irrational fear.

“Limited availability of teammates and reduced support coverage”

This is perhaps the most honest of the arguments, and it deserves a direct response.

The concern is that releasing late on a Friday means fewer people are available if something goes wrong. Response times will be slower. The on-call engineer might be alone.

But notice what this argument assumes: deployments are expected to cause problems that require human intervention. This assumption is so deeply embedded that it passes without comment. It is treated as a law of nature.

It is not. It is a symptom of an immature deployment process.

When deployment is a non-event, when changes are small, well-tested, and automatically rolled back if metrics degrade, you do not need a war room on standby. The system handles problems faster than humans could anyway.

The fear of reduced weekend coverage only makes sense if you expect to need that coverage. And if you expect to need it, that expectation should be treated as a problem to solve, not a constraint to work around.

“Mature engineering teams optimise for reliability, not bravado”

This framing is clever but misleading. It positions Friday deployments as reckless showmanship and the Friday rule as careful professionalism.

The reality is the opposite.

Truly mature teams do not need deployment windows because they have invested in making deployment boring. They deploy dozens of times per day. Each individual deployment is so small and so well-tested that it carries almost no risk. They can deploy on Friday afternoon and go home without anxiety because they have built the systems and practices that make this possible.

Avoiding Friday deployments is not a sign of maturity. It is a sign that you have accepted an immature deployment process as permanent.

The most reliable teams I have worked with never talked about which days were safe to deploy. The question simply did not arise. Deployment was like committing code or running tests: something you do continuously, without ceremony, because your systems are designed to handle it safely.

“Consider blast radius, rollback strategies, observability, and on-call load”

Absolutely. Consider all of these things. But consider them as engineering problems to solve, not as reasons to restrict when you can ship.

Blast radius should be limited by deploying incrementally, using feature flags, and rolling out to small percentages of traffic before going wide. Rollback should be automated and triggered by metric degradation, not by a human noticing a problem and deciding to act. Observability should be good enough that problems are detected in seconds, not hours. On-call load should be reduced by building systems that recover automatically.

If you have all of these things in place, Friday is just another day. If you do not have them in place, the Friday rule is a bandage over a wound that will keep bleeding every other day of the week.

“Good software engineering is about making smart trade-offs”

I agree completely. But the smart trade-off is not between deploying on Friday and waiting until Monday.

The smart trade-off is between maintaining a dysfunctional deployment process and investing in one that actually works. One of these trade-offs solves the problem. The other just delays it until after the weekend.

The expanding forbidden zone

There is a pattern I have seen repeatedly in teams that adopt the Friday rule. First it is “don’t deploy on Friday”. Then it becomes “don’t deploy on Friday afternoon”. Then “don’t deploy after Thursday lunch, just to be safe”. Then “don’t deploy the day before a bank holiday”. Then “don’t deploy during the Christmas period”, which somehow stretches from early December to mid-January.

The forbidden zone keeps expanding because the underlying problem remains unaddressed. You have not made deployments safer. You have just reduced the number of days when you are willing to confront how unsafe they are.

The rule as technical debt

Here is the uncomfortable truth: the “don’t deploy on Friday” rule is a form of technical debt.

Every time you follow it, you reinforce the idea that deployments are inherently dangerous. You avoid the feedback that would push you to improve. You normalise a dysfunctional process instead of fixing it. The rule does not make you safer. It makes you complacent.

If your deployments are scary, the solution is not to do them less often. It is to do them more often, in smaller increments, with better automation, faster feedback, and reliable rollback. The fear you feel about Friday deployments should be treated as a signal, not as wisdom to be enshrined in team policy.

This is an invitation, not a challenge

I want to be clear about what I am not saying. I am not saying “drop everything and push to prod this Friday”. Those of us who encourage Friday deployments are not doing it to be provocative or to score points.

We do it because we have seen what it looks like when teams feel safe enough to do it. We have lived through the pain of not being able to. And we have seen the benefits of moving past that fear.

I am aware of how many invisible blockers stand in the way. Fear of being blamed if something goes wrong. A belief that “this is just how we do things”. Lack of confidence in the release process. Unclear ownership. The absence of good tooling and observability. These are real obstacles, and dismissing them as excuses would be unfair.

But accepting the Friday rule means accepting these blockers as permanent. It means letting fear drive your decisions and treating the status quo as inevitable.

Where the real work begins

So no, you do not need to deploy this Friday. But imagine a world where you could, and it was not a big deal.

Then ask yourself: what would need to be true for that to happen? What stands in the way?

That question is where the real work begins. And answering it honestly will do far more for your reliability than any rule about which days are safe to ship.