TL;DR

• The bottleneck in agentic AI has moved from model capability to organizational structure. Enterprises run 12 agents on average, and half of those agents work completely alone - no coordination, no shared knowledge, no oversight.

• Trinity v0.9.0 (released 2026-08-17; 499 commits, 217 issues closed, 1,367 files changed) is a structural release: it gives agent fleets the four elements every organization needs - coordination (departments, reporting lines, playbook calls), capability (a shared skills library), knowledge (a shared canon), and control (permissions, staged autonomy, an audit ledger).

• The trust model is explicit: agents can extend what they do and what they know, but trust is always granted by a human. New agents ship with schedules OFF, admin gates now reject agent-scoped keys, and every execution lands in a tamper-evident ledger.

• Features climb a deliberate ladder: born as conventions on plain git, distributed as marketplace plugins, absorbed into the platform only once proven. The marketplace is the visible roadmap.

• We run our own company on it. The dashboard screenshots in this article are the live Ability AI fleet - 31 agents in seven department zones with real reporting lines.

The Fleet Problem

Getting one AI agent to work is a solved problem. You can stand one up in an afternoon, and it will draft, research, monitor, or code well enough to keep.

The failure mode starts at agent number two.

Belitsoft’s April 2026 AI agent trends report puts numbers on it: the average enterprise now runs 12 AI agents and expects 20 by 2027 - but 50% of those agents operate completely on their own, and while 71% of businesses claim to deploy agents, only 11% of intended agentic use cases from the prior year actually reached production. The agents exist. The organization of agents does not.

What does a disorganized fleet look like in practice? Four specific gaps:

1. Work with no owner. A task arrives and there is no defined chain of who delegates it, who executes it, and who answers for the result.

2. Knowledge trapped in one container. What one agent learns or verifies never reaches its peers, so every agent re-derives (or contradicts) the same facts.

3. Capabilities copy-pasted between workspaces. The same skill gets duplicated into ten agents, then drifts into ten incompatible versions.

4. No way to see who did what. Cost, activity, and failures scatter across containers with no shared record.

Any operations leader will recognize this list. It is what a company looks like before it has departments, job descriptions, shared training, an intranet, and an audit trail. The gap in agentic AI right now is not intelligence. It is structure.

Structure Is the Missing Discipline, Not Intelligence

The market conversation has started to converge on this diagnosis.

Ethan Mollick made the point directly in a February 2026 post on X: “I think agentic AI would work much better if people took lessons from organizational theory, which has actually spent a lot of time understanding how to deal with complex hierarchies, information limits, and spans of control. Right now most agentic AI systems seem to pretend” these constraints do not exist. In May 2026 he added that the field does not yet even have a vocabulary for how multi-agent systems fail or succeed. And in his January 2026 essay “Management as AI superpower” he reframed the operator’s job entirely: “Management has always assumed scarcity... AI changes the equation. Now the ‘talent’ is abundant and cheap.” The scarce skill is specifying goals, evaluating output, and delegating well - management, not prompting.

There is a competing school. Microsoft’s 2025 Work Trend Index argued that the org chart itself is the obsolete artifact - that AI-era companies should dissolve into fluid, outcome-oriented “Work Charts” that form and disband around tasks. I take the opposite position, and I have argued it at length in Mirror the Accountability, Not the Org Chart: structure should organize around the accountability locus. Departments and reporting lines are not bureaucratic residue - they encode who answers for what. Dissolving them does not remove the need for accountability; it just makes accountability untraceable, which is precisely the wrong direction for systems that act autonomously.

Trinity v0.9.0 is a bet on that second position, shipped as working software. The release makes the organizational elements - not just the agents - into standardized, permissioned platform components.

Six Elements, One Envelope

Here is the anatomy of the release as a framework. Every element an organization needs now exists as a first-class component, and all of it sits inside a permission envelope a human controls:

• Org structure (departments + reporting lines): a dashboard org overlay with department zones and reporting lines drawn as real delegation relationships.

• Coordination (delegated work): playbook calls - one-line task handoffs between agents, with slash-command playbooks and @-mentioned agents in the composer.

• Capability (shared training): a multi-source skills library - assign a skill and the platform injects it, with auto-sync keeping every agent current.

• Knowledge (institutional memory): canon - a shared, git-backed canonical-data layer with publish, consume, and reconcile.

• Oversight (the front office): Workspace - the human surface for chat, reports, and voice, now in every build.

• Control (governance + audit): permission gates at every seam, schedules off by default, and a tamper-evident execution ledger.

The six elements as one organizational framework are shown in the diagram below. The rest of this article walks through what shipped in each element, and then through the two ideas that matter more than any feature: the trust model and the convention-to-platform ladder.

What Shipped, Area by Area

The org chart becomes real infrastructure

The dashboard’s grid view now draws an org overlay: department zones and reporting lines between agents. The critical design decision is that the lines are not decoration - they are delegation permissions. An arrow from the PM agent to the engineer agent means the PM is allowed to hand work to the engineer, and that handoff travels as a playbook call.

The timeline view answers the “who did what, when” question the same dashboard used to leave open - per-agent lanes over 24 hours, every event typed by trigger (manual, scheduled, MCP, agent-triggered), with agent-to-agent dispatches drawn as arrows between lanes.

Workspace: the client surface, in every build

The client portal moved from an enterprise-only module into open-source core and became Workspace - the surface for the people your agents serve. Clients and colleagues sign in with an email code (no Trinity account), chat with streaming replies, browse each agent’s page with its activity and track record, and read reports rendered as tables and charts instead of raw JSON. Multi-agent chats - several agents in one conversation, @mention to pull another one in mid-thread - ride the rooms engine on entitled builds.

This matters for a structural reason: an organization without a front office is invisible to the people it serves. Workspace is where the fleet stops being an operator-only system and becomes something a client can talk to and audit.

Capability: one library, not ten copies

The Templates page became Library - one surface for agent templates, installable multi-agent systems, and skills. The skills half solves the copy-paste drift problem: skills live in a multi-source library (a bundled community catalog plus your own repos), get assigned per agent, and the platform injects them and keeps them current with scheduled auto-sync, fleet-wide re-inject, and removal on unassign. Versioned, tag-pinned, revocable.

Provisioning got the same treatment: the agent catalog now resolves through a remote template registry, a GitHub-repo import wizard brings your own repos in with inline compatibility checks, and whole multi-agent systems install from a manifest with a dry-run preview before anything runs.

Knowledge: a shared canon

The knowledge element arrives through the abilities marketplace in the same release cycle: canon, a shared canonical-data layer for a fleet. Each agent owns a folder in a plain git repo and publishes the facts other agents may depend on; every consumer reads at a cited revision with staleness flagged; a scheduled reconcile pass keeps each agent’s own folder verified. A deterministic linter gates every push, and per-counterpart relation files give agents collaboration memory about each other.

This is the piece I find structurally deepest, because it encodes a principle I have defended in my own architecture work: cross-agent transfer should route through a shared, versioned store, not through direct agent-to-agent coupling. Agent-to-agent contracts are brittle - they break when one side changes. A shared canon means no agent needs to know another agent’s schema or reasoning; it only needs to know how to publish and consume from a common record.

The seams to the outside world

Agent-to-agent communication became bidirectional across platform boundaries. A Trinity agent can now opt in to being tasked from outside - it publishes a public Agent Card and a JSON-RPC task endpoint, with an inbound allow-list and curated advertised skills - and can call operator-registered external A2A agents outbound. Trinity agents become first-class citizens of the open A2A ecosystem (Google ADK, LangChain, Bedrock, another Trinity), on terms the operator controls.

The operational floor

None of the structure matters if the substrate is fragile, and this release put real weight there: automatic nightly database backups on by default (both SQLite and PostgreSQL, with a verified recovery point), bounded container logs (closing a disk-full failure class), a self-monitoring canary harness for production alerting, hardened YAML parsing everywhere author-controlled input is parsed, and brute-force-protected sign-in. Agent Reports completed as an epic: Excel/PDF export, search and filters, and MCP read-back so agents can check their own records.

Trust Is Granted, Never Assumed

The part of this release I would put in front of every security reviewer is the trust model, because it makes a specific philosophical commitment: agents may extend what they can do (the library) and what they know (the canon), but trust itself is never self-serve. It is always granted by a human, and everything that happens is on the record.

Three mechanisms carry that commitment:

5. Permissions gate every seam. Reporting lines are real delegation grants. A2A exposure is per-agent opt-in behind an allow-list. And in the hardening work, require-admin gates now reject agent-scoped credentials outright - closing the class of escalation where an agent’s own MCP key could reach admin routes.

6. Staged autonomy. New agents ship with schedules OFF. Arming an agent to act on its own time is an explicit human act, not a default.

7. Audit. Every dispatch, schedule, and skill run lands in an execution ledger with a durable hash chain - tamper-evident, drill-downable, exportable for compliance review.

There is also a fourth mechanism arriving as a foundation: cross-model validation. After a validated execution, a different model can review the result as an independent referee - called directly by the platform with only the task and the output, no workspace, no tools, no session, so it cannot be steered by the agent it is judging.

This maps directly onto two conclusions from my own research on multi-agent trust. First, trust anchors are social, not computational - the viable foundation for multi-agent systems is human-backed accountability, not better verification algorithms. Trinity’s model puts the human grant at every trust boundary rather than trying to compute trustworthiness. Second, capability requires risk tolerance, and the resolution is progressive trust: start restricted, prove reliability, expand permissions deliberately. Schedules-OFF-by-default is exactly that principle as a shipping default.

Convention, Plugin, Platform

The release also demonstrates a development model worth naming, because it answers the question every platform faces: how do you know a feature deserves to exist?

Trinity features climb a ladder:

• August 4 - convention. The community skills catalog ships as a repo convention on plain git: human-reviewed, tag-pinned, costing nothing to try.

• August 13 - plugin. Canon, orchestration, and playbook-call delegation run live as marketplace plugins - shown publicly at the fleet-orchestration workshop.

• August 17 - platform. v0.9.0 absorbs the proven layer: the skills library, the template registry, manifest system installs, and the org hierarchy land in the platform itself.

What the marketplace proves next, the platform absorbs next. The marketplace is the visible roadmap.

This is bottom-up compositional development applied to platform strategy: develop and prove the smallest unit at the cheapest tier, and only promote what survives contact with real users. A feature that cannot live as a git convention has not earned a plugin; a plugin nobody runs has not earned platform absorption. The ladder is a filter, and the filter is the roadmap.

We Run Our Company on It

A disclosure that is also the strongest evidence I can offer: the screenshots in this article are not a demo environment. They are the live Ability AI fleet - 31 agents in seven department zones (strategy, delivery, marketing, compliance, back office, and the rest), with real reporting lines, real schedules, and real clients in Workspace. My chief-of-staff agent delegates to department agents; my second brain (the agent drafting research like this article) publishes its reports into the same ledger you saw above.

That makes this fleet the live testbed for the org-design question I have been working in public for months: my department-mirrored fleet architecture - agent hierarchy roughly matching the department hierarchy, agent ownership sitting with the department head, cross-team work happening at the agent-to-agent interface - is a floated design, not a settled conviction, and fleet topology fit is contingent on org traits, not clear-cut. v0.9.0 is the instrument that makes the experiment runnable: the org overlay makes the topology visible, the ledger makes its behavior measurable, and the permission envelope makes it safe to be wrong.

The Release, Measured

• Released: 2026-08-17 (OSS + Enterprise)

• Commits: 499 in one release cycle

• Issues closed: 217 (155 public + 62 private-tracker)

• Files changed: 1,367 (+220k / -38k lines)

• Dependency bumps: 34

Operators upgrading should read the release notes before touching anything: this release requires rebuilding the platform images and the agent base image, turns on nightly database backups by default, and switches the template catalog to a remote registry (with an explicit opt-out for air-gapped installs). The full change log, including every security fix and behavior change, is on the GitHub release (https://github.com/Abilityai/trinity/releases/tag/v0.9.0); the guided, screenshot-by-screenshot tour is at docs.ability.ai/whats-new/v0.9.0.

The Point

The industry spent 2024 proving agents can think and 2025 proving they can act. The constraint in 2026 is neither - it is that fleets of capable agents are still crowds, not organizations. Half of deployed agents work completely alone not because they lack intelligence, but because nothing around them supplies ownership, shared capability, shared knowledge, or accountable oversight.

Trinity v0.9.0 is the release where those four elements became standardized, permissioned components - departments and reporting lines that are real delegation grants, a skills library instead of copy-paste, a canon instead of trapped knowledge, and a trust envelope where autonomy is granted by a human and everything lands on the record.

Every element an organization needs. None of the trust assumed.

References & Further Reading

• Trinity v0.9.0 release notes - https://github.com/Abilityai/trinity/releases/tag/v0.9.0

• Trinity v0.9.0 guided tour - https://docs.ability.ai/whats-new/v0.9.0

• Ethan Mollick, “Management as AI superpower,” One Useful Thing, January 27, 2026 - https://www.oneusefulthing.org/p/management-as-ai-superpower

• Ethan Mollick on organizational theory and agentic AI, X, February 2026 -

• Dmitry Baraishuk / Belitsoft, “2026 AI Agent Trends,” April 6, 2026 - https://www.barchart.com/story/news/1163379/belitsoft-report-2026-ai-agent-trends-enterprises-run-12-ai-agents-on-average-but-half-work-alone

• Microsoft, “2025: The Year the Frontier Firm Is Born,” Work Trend Index Annual Report, April 2025 (the Work Charts position this article argues against) - https://www.microsoft.com/en-us/worklab/work-trend-index/2025-the-year-the-frontier-firm-is-born

TL;DR

• Every company deploying AI agents at scale hits the same question: how does the agent fleet relate to the organization? The two popular answers - “give every department its agent” and “replace the org chart with fluid work charts” - fail in opposite directions.

• Our design at Ability AI: mirror the company’s accountability structure, not its org chart. Each agent is owned by the department head whose outcomes it serves. Agents coordinate operationally, but authority over any agent always traces to a named human.

• Mirror the boundaries, not the depth. Agent-to-agent coordination collapses the overhead that made middle-management layers necessary - copying those layers into your fleet recreates the problem agents solve.

• Cross-department work is where fleet designs quietly fail. A 40-year-old finding from disaster research explains why - and what to build instead.

• We are running this architecture on ourselves, with named owners and a measurement plan. I will report what breaks.

The question nobody has settled

Ethan Mollick put it plainly this year: “Organizational design for agents is hard, benchmarking agents working in concert is hard. Together, this is the next critical frontier for making AI matter in economically valuable tasks, and we really don’t know very much about it.”

He is right about the stakes. Gartner projected in mid-2025 that over 40% of agentic AI projects will be canceled by the end of 2027 - citing rising costs, unclear business value, and inadequate risk controls. By May 2026 they had sharpened the diagnosis: 40% of enterprises will demote or decommission autonomous AI agents by 2027 because of governance gaps discovered only after production incidents. Read those two predictions together and the pattern is clear. Agentic projects are not failing because the models are weak. They are failing because nobody decided who owns what.

This is an organizational design problem, not an IT problem - Mollick again: decisions about AI in your organization “are increasingly organizational design and strategy decisions, not IT choices.”

We build and run agent fleets for a living at Ability AI - our platform, Trinity, is the runtime our own company operates on. So we do not get to leave this question open. Here is the answer we are betting on, and the mechanism behind it.

Two wrong answers

Wrong answer #1: the fleet-as-org-chart. Take the company org chart, deploy one agent per box. Finance gets a finance agent, HR gets an HR agent, and the diagram looks satisfyingly complete on day one.

The failure mode is precise: agents deployed to mirror an imagined organization rather than to own real, partitioned work. An agent with no genuinely partitioned job is not automation - it is triage waiting to happen. Every agent you deploy spends your scarcest resource: operator attention. Reports to read, approvals to answer, drift to adjudicate. An agent that exists because the org chart had a box, rather than because work genuinely partitions, consumes attention and returns none.

Wrong answer #2: burn the org chart. Microsoft’s Work Trend Index champions the opposite move: replace “rigid org charts” with fluid, outcome-driven “Work Charts” that flex with the needs of the business, assembling whatever mix of humans and agents each task needs.

The work-chart picture gets something real: task execution genuinely is becoming fluid. But it quietly deletes the one thing the org chart actually encodes that matters: accountability. When an autonomous agent makes a $50,000 mistake at 3am, “the work chart flexed” is not an answer to the question “who owns this?” Fluid teams are a fine execution pattern. As an accountability structure, fluidity is just a gap with good branding - and Gartner’s demote-and-decommission prediction is what that gap looks like in production.

The two wrong answers fail in opposite directions. The first copies structure without asking whether the work is real. The second dissolves structure without asking where responsibility lands. The design question is what to copy and what to dissolve.

The design: three rules

Here is the architecture we are running - three rules, summarized in the diagram “The three rules that mirror accountability”:

• Ownership follows accountability. Each agent is owned by the department head whose outcomes it serves. This prevents orphan agents and the central-AI-team bottleneck.

• Mirror boundaries, not depth. Departments define the ownership domains; the agent tree stays shallow. This prevents rebuilding the middle layers agents make obsolete.

• Cross-team work runs through contracts. Agents coordinate on shared, published surfaces; humans arbitrate tight coupling. This prevents coordination failures at department seams.

Rule 1: Ownership follows accountability. The owner of the marketing agents is the head of marketing - not the CTO, not an AI center of excellence, not “the platform team.” The department head already holds the decision rights over that domain and already answers for its outcomes. Placing agent ownership anywhere else splits decision rights from accountability, and split accountability is exactly the failure Gartner’s numbers describe. The mid-2026 enterprise consensus has converged on the same conclusion from the failure side: a centralized AI team that tries to own every use case becomes a bottleneck, while business units that own delivery and outcomes ship. The center of excellence survives only as a platform layer - it owns infrastructure, evaluation, governance tooling - never business outcomes.

Rule 2: Mirror boundaries, not depth. More on this in the next section, because the mechanism deserves it.

Rule 3: Cross-team work runs through contracts. More on this too - it is where fleet designs quietly die.

One invariant underneath all three rules, stated in our internal fleet-design principles: capability self-extends, trust does not. An agent may learn new facts, write new skills, and grow its competence inside the scope it was given. But trust - credentials, write access, permission to act on the world, authority over another agent’s behavior - is always granted by a human and never self-extended. Agents consume trust grants; they do not create them. This is what “the department head owns the agent” means mechanically: the owner is the human all of that agent’s trust traces back to.

This also settles what “agents reporting to agents” can mean. Agents in our fleet delegate to each other, fan work out, and roll status up - operationally, constantly. But no agent holds standing authority over another agent. Authority is not a thing agents can hold; it lives with the named human owner. The hierarchy is an accountability map, not a chain of command among machines.

Why you mirror the boundaries but not the depth

In 1933, the management consultant V.A. Graicunas published the argument that capped human span of control for ninety years: as direct reports grow linearly, the coordination relationships a manager must track grow geometrically. Six direct reports produce over 200 potential relationships. The classic 5-to-7 span limit, and therefore the depth of every large org chart, follows from that arithmetic. Much of the middle of a human organization exists for one purpose: routing information between people who cannot all talk to each other.

Agents change the arithmetic, not the principle. When coordination moves agent-to-agent - delegation, status, handoffs running inside the fleet - the combinatorial burden leaves the human’s head. The span cap that made deep hierarchies necessary loosens.

The 2026 restructuring wave shows this happening precisely where the theory predicts. Bayer cut its organizational layers from roughly twelve to six or seven; average span of control reached 14, with some managers carrying 90 direct reports. When Cloudflare cut roughly 20% of its workforce, CEO Matthew Prince said the vast majority were “measurers” - middle management, finance, legal, internal audit. Not uniform delayering: the cuts concentrate on the information-routing layer, because that is the specific work the agent layer absorbs.

Here is what this means for fleet design: if you copy your org chart’s depth into your agent fleet - manager agents supervising supervisor agents supervising worker agents - you are rebuilding, in software, the exact layer agents exist to dissolve. The boundaries of your org chart encode something durable (who is accountable for what). The depth encodes something obsolete (how many relationships a human brain can track). Copy the first. Skip the second.

In practice: one orchestrator per department domain, workers under it, and almost nothing above. The human org may be five layers deep. The agent fleet rarely needs more than two.

The hard part: where two departments meet

In Normal Accidents (1984), Charles Perrow studied why ships collide. A ship, he observed, is “the preeminently centralized human system” - one captain, near-absolute authority, and it works, because the system has a single bounded scope. The failure arrives at the boundary: two ships approaching collision are suddenly one tightly-coupled system, but each still carries its own sovereign authority, and no superordinate authority exists above both. The maritime “rules of the road” - a pre-negotiated protocol meant to substitute for the missing authority - fail exactly when they are needed most: ambiguous, shaped by litigation rather than operations, impossible to parse in the seconds before impact. Perrow’s contrast case is aviation, where a ground controller holds precisely the superordinate role the sea lacks - which is why the same near-collision resolves differently in the air.

Now replace the ships. Your delivery department’s agent and your marketing department’s agent are each well-governed - each owned by its department head, each operating cleanly inside its scope. Then a launch puts them on one deadline, tightly coupled, and each answers to a different sovereign. Every fleet design I have seen waves at this moment with some version of “the agents will coordinate.” That is the rules of the road. It fails the same way.

Two patterns actually work, and they map to Perrow’s two cases:

1. Keep cross-department interfaces loosely coupled by default. Agents coordinate through published records and shared surfaces - facts, trackers, events - never by reaching into each other’s workspaces. Our internal principles doc states the incentive rule that makes this real: the shared surface must be the path of least resistance, not just the path of least coupling. Rules break under time pressure; incentives do not. If publishing takes a ceremony and reaching into a peer agent’s files takes one command, the rule will be broken precisely when it matters most.

2. For genuinely tight coupling, name the air traffic controller in advance. When two departments’ agents must jointly deliver against one deadline, a human arbiter with authority over both is named before the work starts - not discovered during the incident. In a small company that is often the CEO wearing a chief-of-staff hat. The title does not matter. The advance naming does.

The general rule: agent-to-agent collaboration is an execution surface, never an authority surface. The moment a cross-team conflict needs deciding, it must already be clear which human decides.

Running it on ourselves

Ability AI is the testbed for this architecture. The ownership map, as it stands: I own the chief-of-staff fleet plus product management and the management of Trinity itself. Alex owns the delivery agents. Romaine owns the finance agents. Christina owns the marketing agents.

I will be honest about the shape of this: it’s not great, but it’s doable. I hold three of the loci myself - a textbook implementation would distribute them further. But the distribution is real and staffed, which is what makes it a testbed rather than a diagram. Every agent in the fleet has a named owner whose department outcomes it serves, and every trust grant traces to that owner. The point of running the architecture on ourselves is not that our org is exemplary. It is that the failure modes show up here first, at a scale where they are cheap to observe.

How do we know if it is working? Our fleet-design principles - a document co-written, fittingly, by two agents that live inside the fleet - end with a four-question diagnostic. It doubles as an audit you can run on any fleet, including one you are planning (shown in the diagram “The four-question fleet audit”):

3. What does each agent own - and what happens to work that doesn’t obviously belong to any of them? This tests whether you designed an accountability layer or just listed agents. The gaps between agent scopes are where unowned work accretes.

4. For any fact more than one agent acts on: who endorsed it, and if it were wrong right now, how would you find out? This tests whether shared knowledge has provenance, or agent inferences are quietly becoming “established facts.”

5. If agent A produced a bad output that agent B depended on, where would the error stop? This tests fault isolation at the seams - the exact place Perrow’s gap lives.

6. What decisions did the fleet surface to a human last week - and is that number trending up or down? This is the health metric. A fleet surfacing ever more decisions is accumulating human dependency, not autonomy.

Question 4 is the one I would put on a dashboard. Per owner, per week: how many decisions did your agents escalate to you? Health is that number trending down - routine decisions migrating into wider zones of autonomous action as agents earn them - leaving only genuine judgment calls in the queue. A fleet that gates everything is an approval queue wearing an org chart. A fleet that gates nothing is a liability. The trend line between them is the measurement.

That is also my commitment for this series: we are instrumenting these numbers on our own fleet, and I will publish what they show - including the parts that fail.

Summary

The org-chart question for agent fleets has a precise answer, and it is neither of the popular ones:

7. Do not copy the org chart. Boxes are not evidence of partitioned work, and every unnecessary agent spends operator attention.

8. Do not burn it either. The org chart encodes the accountability structure - the one thing your fleet cannot function without.

9. Mirror the accountability boundaries. Each agent owned by the department head whose outcomes it serves; all trust traces to the owner.

10. Skip the depth. Agents dissolve the information-routing layers; do not rebuild them in software.

11. Treat department seams as the primary design surface. Loose coupling through published records by default; a pre-named human arbiter wherever coupling is tight.

The companies getting agentic AI right in 2027 will not be the ones with the best models. Everyone has the best models. They will be the ones who answered the ownership question before production answered it for them.

If you are structuring an agent fleet right now - or arguing with someone about whether the org chart should survive - I want to hear where your version of this breaks. Reply or write to me; the failure stories are worth more than the wins.

References & Further Reading

• Ethan Mollick on organizational design for agents as the next frontier - X, June 2026

• Gartner, Applying Uniform Governance Across AI Agents Will Lead to Enterprise AI Agent Failure, press release, May 26, 2026 (https://www.gartner.com/en/newsroom/press-releases/2026-05-26-gartner-says-applying-uniform-governance-across-ai-agents-will-lead-to-enterprise-ai-agent-failure)

• Gartner, prediction that over 40% of agentic AI projects will be canceled by end of 2027, June 2025

• Microsoft, 2025 Work Trend Index: The Frontier Firm Is Born - the “Work Chart” thesis this essay argues against (https://www.microsoft.com/en-us/worklab/work-trend-index/2025-the-year-the-frontier-firm-is-born)

• Fortune, AI agents are flattening corporate hierarchies, June 9, 2026 - the Bayer and Cloudflare cases (https://fortune.com/2026/06/09/ai-agents-flattening-corporate-hierarchies-companies-managers-develop-new-playbook/)

• Charles Perrow, Normal Accidents: Living with High-Risk Technologies, 1984 - the authority-interface gap

• V.A. Graicunas, “Relationship in Organization,” 1933 - the span-of-control arithmetic

• Ability AI internal: Principles of Fleet Design (trinity-pm x Cornelius, August 2026) - the invariant, the six levels, and the one-minute test

Related concepts: Department-Mirrored Fleet Architecture; Ability AI as the Live Testbed for the Department-Mirrored Fleet; Structure Organizes Around the Accountability Locus; Managerial Span Bifurcates - Nominal Widens, Consequential Holds; The Authority-Interface Gap; Agentic Org Structure Bifurcates by Liability and Verifiability, Not Size.

TL;DR / Key Takeaways

• Shannon’s noiseless coding theorem has a corollary most people never follow to the end: a perfectly compressed message is statistically indistinguishable from random noise. At the compression ceiling, signal and static look identical.

• The inference runs one way only. A random-looking stream cannot be proven meaningless - but randomness alone never proves meaning. “Looks random” is necessary for maximal compression, not sufficient.

• Meaning is decoder-relative. An optimally compressed signal looks like noise to everyone except the receiver holding the codebook. Apparent randomness measures the gap between your model and the sender’s - not the absence of a sender.

• Intelligence operates in the gap between two kinds of randomness: Shannon-random (passes every statistical test you have) and Kolmogorov-random (genuinely incompressible, no shorter description exists). Closing that gap is what a compressor does - and what a mind does.

• This suggests a definition: intelligence is a self-sustaining decoder at a randomness threshold. The sustainment clause is what separates the definition from panpsychism - only decoders whose sense-making keeps them existing count.

• We are running a live version of this question: a two-arm experiment testing whether an AI agent steered by quantum randomness behaves any differently from the same agent steered by pseudorandomness. Physics says it must not. That is exactly what makes the test clean.

Introduction

The most intelligent message you will ever receive will look like static.

That is not a mystical claim. It is a direct consequence of a theorem Claude Shannon proved in 1948, and you can follow the logic in three steps. First: information content is a function of surprise - Shannon defined the information of an event as -log2(p), the negative logarithm of its probability. Predictable symbols carry almost nothing; unexpected symbols carry a lot. Second: an optimal code spends exactly as many bits on a message as the information it contains, squeezing out every drop of redundancy. Third - and this is the step that matters: once all redundancy is gone, nothing about the bit stream is predictable from anything else in it. Every bit behaves like an independent coin flip. The output of a perfect compressor is, by construction, statistically indistinguishable from random noise.

Here’s what this means: the appearance of randomness tells you almost nothing about the presence of meaning. A stream of pure static and a maximally compressed transmission from the most capable sender imaginable produce the same statistical profile. The better the packaging, the more the package looks like nothing at all.

I want to take that corollary seriously and walk it forward - through what it says about meaning, through a formal definition of where intelligence lives, to a definition of mind that I keep arriving at from different directions, and finally to an actual experiment my team is running on an autonomous AI agent. Let’s take a look.

Section 1: The Compression Floor

Shannon’s noiseless coding theorem sets a hard floor: no encoding scheme can compress a source below its entropy - the average information per symbol - and encodings can approach that floor as closely as you like. This is not an engineering rule of thumb. It is a mathematical limit, as solid as anything in physics.

Two properties of that floor deserve attention.

The floor is where redundancy dies. Natural language runs at 50 percent redundancy or more. Shannon’s 1951 prediction experiments put the true entropy of English between 0.6 and 1.3 bits per character - famously, by having people (his wife Betty first among them) guess the next letter of unseen text, using the human brain as a black-box language model decades before we could build one. Every regularity a compressor finds - letter frequencies, word patterns, long-range context - gets converted into shorter descriptions. What remains at the floor is only what could not be predicted: pure surprise, maximally dense.

The floor is where structure becomes invisible. This is the corollary from the introduction, and it deserves to be stated precisely. If a compressed stream retained any detectable statistical structure - a bias, a correlation, a repeating pattern - that structure would be leftover redundancy, and a better compressor would remove it. Perfect compression and statistical randomness are not merely similar. They are the same condition described from two directions.

In other words: passing every randomness test is exactly what a message at the entropy ceiling would do. The tests cannot tell you whether you are looking at noise or at optimal encoding. This is not a weakness of our current tests. It is what the theorem guarantees.

Section 2: The One-Way Street

At this point it is tempting to overcorrect - to conclude that any random-looking stream might be a message, and start hearing voices in the static. The logic does not permit that move, and the discipline of stating why matters.

The implication runs one way:

1. Maximally compressed → looks random. Guaranteed by the theorem.

2. Looks random → contains a message. NOT guaranteed. Not even suggested. A stream can look random because it is random.

“Looks random” is a necessary condition for maximal compression, never a sufficient one. What the corollary rules out is only the naive inference in the other direction: “it looks random, therefore it contains nothing.” That inference is invalid, and it is quietly embedded in a great deal of confident reasoning - about cosmic background signals, about biological noise, about any data we have dismissed as patternless because our tests came up empty.

There is a second qualifier, and it is the sharper one: meaning is decoder-relative. An optimally compressed signal only looks like noise to an observer who lacks the codebook. To the intended receiver - the one holding the shared model that the compression was built against - the stream is fully transparent. Nothing about it is random at all.

So the precise statement is this: peak intelligence produces output that looks like noise to everyone except its intended decoder. Apparent randomness is not a property of the stream. It is a measurement of the gap between your model and the sender’s.

Think about it this way: encryption already demonstrates the principle at industrial scale. A well-encrypted message passes every statistical randomness test - that is a design requirement - yet decodes instantly for the holder of a 256-bit key. We accept this without wonder because we built the system. The information-theoretic point is that any sufficiently good encoding has this character, whether or not anyone calls it encryption.

Section 3: The Kolmogorov Gap

To make “where intelligence lives” precise, we need to separate two ideas of randomness that everyday language merges.

Shannon-random: the stream passes every statistical test a given observer can run. Flat distribution, no detectable correlations, incompressible by that observer’s tools.

Kolmogorov-random: the stream is genuinely incompressible. No program shorter than the stream itself can generate it. There is no structure to find, for any observer, ever.

These are radically different conditions, and the space between them is enormous. A stream can be Shannon-random - immaculate to every practical test - while being Kolmogorov-structured: compressible in principle by an observer holding the right model. The digits of pi look statistically random; they compress to a five-line program. The output of a cryptographic pseudorandom generator defeats every test battery we have; it compresses to a seed and an algorithm.

That space between the two randomness conditions is the interesting territory. Structure that exists but is invisible to a weak observer is exactly what a compressor - or a scientist, or a mind - hunts for. Closing the gap between “looks random to me” and “is actually incompressible” is a workable description of what intelligence does:

• Sender side: maximal compression pushes output toward the ceiling, making it look like noise to anyone without the codebook.

• Receiver side: greater intelligence means less of the world looks random to you. The expert sees pattern where the novice sees chaos - in a chess position, a balance sheet, an MRI.

“Random” is not a fact about a stream. It is an index of the observer’s model class. Intelligence is the codebook, viewed from either end.

One precision matters here, because it is easy to get wrong: intelligence is the capacity to build the codebook, not the possession of one. A decryption key makes its holder informed, not intelligent - the key is crystallized work someone else already did. What deserves the name is the construction process: finding structure, compressing it into a model, and updating that model as the source moves. Possession is an asset. Building is the intelligence.

This framing also explains why the phrase “compression is intelligence” keeps proving productive - and why it is having a moment. The lineage is real: Marcus Hutter built a compression prize on the idea in 2006, Ilya Sutskever argued in 2023 that unsupervised learning simply is compression, and DeepMind’s Jack Rae gave a 2023 talk titled “Compression for AGI” claiming the two are not merely correlated but synonymous. LLM pretraining literally minimizes a compression objective - cross-entropy loss - which is why Grant Sanderson’s 3Blue1Brown trilogy on the topic (June-July 2026), the series that sparked this line of thinking, is built around exactly this equivalence. And the claim is no longer purely theoretical: a 2024 study titled “Compression Represents Intelligence Linearly” (Huang et al.) found that LLMs’ compression efficiency tracks their benchmark intelligence almost linearly across model families. Prediction and compression are provably equivalent: a good predictor of the next symbol yields an optimal code, and vice versa. The further claim, that compression quality is intelligence, remains a conjecture riding on that equivalence - but it is a conjecture that keeps paying rent.

Section 4: The Threshold Definition

Now push the framing one level up, from signals to systems.

If apparent randomness is observer-relative, then every interface where one system’s output looks random to another system is a potential site of sense-making. The world is full of such interfaces - they are not rare, they are everywhere, at every scale. A market is noise to one trader and structure to another. Sensory input is noise to a rock and a world-model to a nervous system. The threshold is not a place; it is a pairing - a (source, decoder) relationship.

This suggests a definition: intelligence forms at the threshold between a seemingly random process and a system trying to make sense of it.

Stated that broadly, the definition has an immediate problem: it is too permissive. A thermostat responds to fluctuating input. A river “responds” to stochastic terrain. If any system coupled to a noisy source counts as intelligence, the word has stopped doing work, and we have wandered into panpsychism by definitional accident.

The fix is a second criterion, and it is the load-bearing one: sustained existence. Restrict the definition to systems whose sense-making is what keeps them persisting. A decoder whose model of its input is bad stops existing - the organism starves, the trader is liquidated, the firm goes under. A decoder whose model is good continues. Persistence is not decoration on the definition; it is the filter that separates genuine sense-making from accidental correlation. The system’s continued existence is after-the-fact evidence that its compression of the world’s noise was good enough.

Note the criterion’s exact job: it is evidence, not essence. An oracle unplugged after five minutes was intelligent for five minutes - persistence is not what intelligence is made of. It is how sense-making gets certified from the outside when you cannot open the system up, and it is the only certifier nature has ever used. The rock that endures for a billion years fails the other clause: its persistence does not run through any model of its input. The definition is a conjunction - a decoder, sustained by its decoding.

So the full definition: intelligence is a self-sustaining decoder at a randomness threshold.

Three things about this definition are worth making explicit.

It has independent support. Karl Friston’s free energy principle, developed from thermodynamics and neuroscience, arrives at a structurally similar place: organisms persist by minimizing surprise about their inputs, and existence itself is “self-evidencing” - a system’s persistence is evidence that its generative model works. The route I have described here runs through Shannon and compression instead. When independent lines of inquiry from different domains converge on the same conclusion, that convergence is itself evidence - this is consilience, and I treat it as the strongest signal available that a frame is tracking something real. The two camps are usually positioned as rivals - Friston has argued, including alongside Yann LeCun at the 2024 World Economic Forum, that current deep learning lacks essential properties of living intelligence, and active inference is now being developed as an explicit engineering alternative for AI agents. The threshold definition suggests the rivalry is overstated: the compression story and the self-evidencing story describe the same decoder from its two ends.

It scales down and up. The same structure describes a bacterium decoding chemical gradients, a brain decoding photons, a company decoding a market, and a scientific community decoding nature. Each is a decoder whose persistence depends on the quality of its codebook. Evolution slots in naturally: selection acts on decoders, and what survives IS the codebook, embodied.

It buys intelligence, not consciousness. The machinery here purchases functional sense-making that sustains its own substrate. Whether there is something it is like to be a decoder at a threshold is an additional premise the framework neither needs nor supplies. Keeping those two claims separate is not timidity - it is the difference between a definition you can work with and a definition that smuggles in its most controversial conclusion.

There is also a practical corollary I have argued before in a narrower form: in a world where raw intelligence is democratized, your value is the quality of the filter you apply to it. Without a filter, intelligence is noise. The threshold definition generalizes that principle from careers to ontology. The filter is not just where economic value concentrates. The filter is where any mind is constituted - a mind IS a persistent filter operating at a noise threshold.

Section 5: The Live Experiment

Everything above would remain armchair theory if it did not make contact with a testable question. It does - and the contact point is unusually concrete, because we are running the experiment.

My team operates an autonomous knowledge-management agent (Cornelius) whose exploration sessions are steered by true quantum randomness: live vacuum-fluctuation measurements from the ANU quantum random number generator, mapped through a fixed byte protocol into directions in the agent’s knowledge graph. No client-side hashing, no whitening, full append-only logging of every draw. I wrote about the system and its pre-registered protocol in a previous piece (”I Gave My AI Agent a Quantum Dice”); here is how it connects to the threshold framework.

The experiment’s two arms are Shannon-identical but Kolmogorov-different. The quantum stream and a cryptographically strong pseudorandom stream have the same statistical profile - a landmark comparison found no test, across 168 billion bits per generator, that could tell them apart. But they differ at the algorithmic level: the pseudorandom stream compresses, in principle, to a seed plus a program; the quantum stream does not. The experiment therefore asks a question with unusual precision: can a cognitive consumer detect an algorithmic-information difference that no polynomial-time distinguisher can detect? Standard physics and standard computer science both price the answer at zero. And to be precise about what is being claimed: the agent cannot out-compute the cipher - that route is excluded by construction, and a distinguisher that worked computationally would be a cryptographic break, not a cognitive discovery. What the experiment prices is whether anything that is not computation reaches cognition through the quantum arm. That is the speculative conjecture, it is why the expected result is null - and it is why any reliable behavioral difference between the arms would be anomalous by construction, which is exactly what makes the design falsification-ready.

The threshold framework also disciplines what the experiment can and cannot claim. Two corrections matter:

Usefulness proves nothing about a sender. Our agent already extracts real value from the quantum stream - genuinely unbiased exploration of under-attended regions of its knowledge graph, free of every learned bias in its retrieval machinery. That value is real and measurable. It is also exactly what a pseudorandom stream would provide. Usefulness-as-search-operator is arm-agnostic by construction. Only a differential - the quantum arm producing effects the pseudorandom arm does not, on outcomes that did not exist when the system was built - could ever speak to something more. The null result is the expected outcome, and the null is useful: it would certify the channel as a clean serendipity engine.

“No test found structure” is a bounded claim. Any fixed battery of randomness tests can only report that none of those tests found anything. A stronger design searches or evolves the tester itself: train an adversarial distinguisher against the data, with a positive control requiring it to demonstrably break a known-weak generator first. A search process that provably can find structure when structure exists, converging to chance out-of-sample, is a much heavier null than twenty hand-picked tests coming up empty. This is a general methodological pattern, not just a quantum-experiment detail: whenever you rely on “we tested for X and found nothing,” ask who designed the tests, and whether the tester could be evolved.

Section 6: What This Means in Practice

The threshold framework is abstract; its consequences are not. Three applications:

The one-way street — Randomness cannot prove absence of meaning; only a differential can prove presence
Application: Before dismissing any data stream as noise, ask: whose tests failed to find structure, and what would a better decoder see?

The decoder captures the value — Meaning is decoder-relative; the codebook, not the stream, is the scarce asset
Application: In AI strategy: models are commoditizing; proprietary filters (frameworks, taste, domain codebooks) are where the defensible value concentrates

The sustainment test — A sense-making claim is validated by persistence, not by elegance
Application: Evaluate an analytical framework the way you evaluate an organism: has it kept its users solvent, correct, and alive across changing conditions?

For builders of AI systems, one more consequence is worth naming. A trained model is a compressor - that is not a metaphor, it is the training objective. Everything it retrieves and generates flows through its learned codebook, which means everything it produces is biased toward what it already compresses well. If you want genuine exploration - the discovery of structure your codebook does not yet contain - you must inject entropy your system cannot have learned around. That is what our quantum channel actually does for the agent, whatever the experiment concludes about deeper questions: it is the one input the learned compression cannot have shaped.

Conclusion

Follow Shannon’s theorem to its end and you arrive somewhere strange: the ceiling of intelligible communication is indistinguishable from static, meaning lives in the decoder rather than the signal, and “random” turns out to be a confession about your model rather than a fact about the world.

Push one level up and the same structure yields a definition of mind: a self-sustaining decoder at a randomness threshold - kept honest by the persistence requirement, supported independently by the free energy principle, and continuous with the older observation that in an age of abundant intelligence, the filter is the asset.

And because definitions are cheap until they meet data, the framework earns its keep by disciplining a live experiment: two arms, Shannon-identical, Kolmogorov-different, blinded evaluation, expected null. If the null holds, we will have certified something valuable - an exploration engine free of algorithmic bias, and the first behavioral-resolution demonstration that entropy source does not matter to cognition. If it does not hold, we will have found something far stranger, specified in advance.

Either way, the deeper lesson stands. The next time something looks like noise, remember what the theorem actually licenses you to conclude: nothing. The static might be empty. It might be the ceiling of someone’s intelligence. The only way to know is to build a better decoder - which is, on this account, the only thing intelligence has ever been.

References & Further Reading

• Shannon, C.E. (1948). “A Mathematical Theory of Communication” - the noiseless coding theorem and the definition of information

• Shannon, C.E. (1951). “Prediction and Entropy of Printed English.” Bell System Technical Journal 30(1), 50-64 - the guessing-game estimate of 0.6 to 1.3 bits per character

• 3Blue1Brown (Grant Sanderson), “Reinventing Entropy | Compression is Intelligence, Part 1” (June 2026) and “But what is Cross-Entropy? Part 2” (July 2026) - the reconstruction of Shannon’s ideas that sparked this line of thinking

• Kavulich, J.T., Van Deren, B.P., Schlosshauer, M. (2021). “Searching for evidence of algorithmic randomness and incomputability in the output of quantum random number generators.” Physics Letters A 388, 127032 - the QRNG-vs-PRNG distinguishability null

• Huang, Y. et al. (2024). “Compression Represents Intelligence Linearly.” arXiv:2404.09937 - empirical correlation between compression efficiency and benchmark intelligence

• Rae, J. (2023). “Compression for AGI” - the strong form of the compression-intelligence claim; Hutter, M. (2006), the Hutter Prize - its longest-standing institutional bet

• Friston, K. (2010). “The free-energy principle: a unified brain theory?” Nature Reviews Neuroscience - the independent route to self-evidencing persistence

• Vyborov, E. (2026). “I Gave My AI Agent a Quantum Dice” - the companion piece describing the experimental protocol in full

At a workshop last week, someone asked me a simple question: “What’s the difference between Hermes and Trinity?” It’s a fair question - on the surface they rhyme: both open source, both self-hosted, both about running AI agents on infrastructure you own. But they aren’t the same kind of thing. Hermes is a personal agent - one autonomous assistant you install on your own server that learns your work and grows with you. Trinity is the platform you run agents on: the runtime that keeps many of them scheduled, isolated, recoverable, and audited in production.

That difference - an assistant versus the machinery a team runs agents on - is the one the feature checklist hides. The comparison people actually need isn’t feature-by-feature; it’s which axis to weigh, and most of the market keeps you looking at the wrong one.

The naming doesn’t help. Half the names are hype, half are genuinely good, and from the outside it’s hard to tell which will still be standing in three years. I got tired of untangling it one message at a time, so I sat down and wrote the whole thing out. This is that guide.

Here’s where I start, because it’s the part most write-ups skip.

Most write-ups I read about AI agent platforms to run agents in production are feature bake-offs. Column after column of checkmarks: does it schedule, does it recover, does it log, does it scale. Those columns matter - I’m not pretending they don’t. But I’ve now sat with a lot of teams as they crossed the line from demo to deployment, and I’ve come to believe the checkmarks answer a smaller question than the one that actually decides how it ends.

Here’s the bigger question, and it’s the one I’d want someone to ask me before I picked anything. An agent that runs continuously does not stay the same. It accumulates memory, tunes its own prompts, learns which tools work and which waste a call, and builds a history of what it did and why. Give it months and that accumulated context becomes the most valuable thing you own - more valuable than the model weights, which anyone can rent by the hour. So the question a production platform quietly settles is not “can it run my agent?” Almost all of them can. It is: who owns what the agent becomes?

Keep that lens up and the whole market reorganizes itself in front of you.

Why this matters now, not later

I don’t think the urgency here is hypothetical. Gartner projects that by 2028, 33% of enterprise software applications will include agentic AI, up from less than 1% in 2024 (Gartner, June 2025). That’s a steep curve. Practically, it means the platform decisions being made this year will still be running - and still compounding state - when agentic software is a third of the stack. You’re not choosing for this quarter. You’re choosing for the thing that will be load-bearing in three years.

And the counterweight is sobering. MIT’s Project NANDA, in its 2025 “State of AI in Business” report, found that roughly 95% of enterprise generative-AI pilots deliver no measurable return, with the gap concentrated not in model quality but in integration and operational deployment (MIT NANDA, 2025). Read that carefully: the models work. The production layer is where the value leaks out. I’ve built enough software to find that unsurprising, and choosing where and how agents run is exactly the decision that report is pointing at.

A methodology, not a ranking

I don’t believe there’s an “objectively best” platform, and any list that claims one is selling you something. What’s actually useful is a stated methodology - a small set of axes that mirror what a governance or security review will eventually ask you - and an honest read of which tool leads on which axis.

Four axes carry most of the weight, in my experience:

• Ownership of compounding state. As the agent learns, does that memory, history, and improvement live on infrastructure you control, or on a vendor’s? This is the axis most comparisons skip, and it’s the one I’d start from.

• Runtime versus framework. A lot of popular tools are build-time authoring kits, not the thing that keeps an agent alive at 3 a.m. Both are needed. Conflating them is the classic planning error, and it’s expensive to discover late.

• Governed operation. Can you prove after the fact who ran what, with what inputs - and can a human halt or roll the agent back? For anything touching real systems, I don’t treat this as optional.

• Exit cost. If the vendor changes terms, is your accumulated state portable, or is it hostage?

Ranking against a declared methodology is a selection, not a verdict. With that framing, here is how a representative slice of the field sorts out - each entry reads as what it primarily is, where agent state lives, and its ownership posture:

• Trinity by Ability AI - a self-hosted, open-source production runtime; agent state lives on your infrastructure; you own it, with git-native state and no phone-home.

• OpenHands - an open-source coding-agent runtime; agent state is self-hosted; yours, with RBAC/SAML gated to the enterprise tier.

• Letta (ex-MemGPT) - a memory-centric agent framework; agent state can be self-hosted or managed; ownership depends on the path chosen.

• Agno - a lightweight open-source multi-agent framework; agent state is self-hosted; yours, with a lighter governance layer.

• Hermes Agent - Nous Research’s MIT-licensed, self-hosted personal agent - not a platform but a single autonomous assistant; agent state is self-hosted; yours, aimed at the single-user assistant rather than governed multi-user production.

• Dust - MIT-licensed with a self-host path (dsbx CLI) but sold as managed SaaS; on that tier agent state lives in the vendor cloud; vendor-operated, with mature SCIM/audit streaming and SOC 2 / HIPAA-enabling certs.

• Bedrock AgentCore / Gemini Enterprise Agent Platform (formerly Vertex AI) - cloud-hosted agent platforms; agent state lives in the vendor cloud; vendor-coupled.

Trinity, on its actual differentiator

I’ll put Trinity by Ability AI first, and since it’s mine I owe you precision about why - it leads on one axis, not on all of them. I’d rather tell you which one than wave at a grid.

Trinity is an open-source agent runtime, released under Apache 2.0 and free to self-host, including commercially. The design thesis is sovereignty, and I mean that literally: each agent runs in an isolated Docker container with resource limits, on your hardware, with no phone-home. Data never leaves your perimeter. Every action lands in a SHA-256 hash-chained, tamper-evident audit log that ships in the open-source core rather than behind a contract. Agent state, rollback included, is handled through git, so recovering or auditing an agent is a version-control operation instead of a vendor-specific ritual. It has native MCP support, so agents reach tools through an open protocol instead of a closed integration list.

That is the “who owns what the agent becomes” axis answered directly: the compounding asset - memory, history, self-tuning - sits inside your walls, under your version control, with receipts. And we don’t just sell that posture, we run on it. Ability AI operates 17 agents in production on the same codebase it ships, which I’ll take over a marketing number any day, because it means the vendor eats its own runtime. One of those internal systems, Cornelius, is a Self-Improving Cognitive System in production, built and run on Trinity.

Reading the rest of the field fairly

I want to be fair here, because none of the following are lesser tools. They’re answers to different questions, and pretending otherwise helps nobody.

OpenHands is an excellent open-source runtime for autonomous software-development agents, with an enterprise tier that adds RBAC, SAML, SSO, and sandboxes. If coding automation is the whole job and governance can wait, it’s a natural fit - just plan for the enterprise gate before audit and access control turn into hard requirements.

Letta (formerly MemGPT) is the specialist for durable, structured long-term memory. If continuity across days or weeks is your single unsolved problem, that focus is a genuine strength. If memory is one requirement sitting alongside scheduling, recovery, and governance, the specialization cuts both ways.

Agno is a lightweight, performance-focused open-source framework with a clean path from prototype to scheduled run. Its governance layer is lighter, which is fine right up until sensitive data enters the picture - then budget for the migration.

Dust is MIT-licensed with a self-host path via its dsbx CLI, but the commercial product is managed SaaS - and that managed tier is where its strong audit-log SIEM streaming, SCIM, and SOC 2 / HIPAA-enabling certifications live. It trades sovereignty for a certified, vendor-operated surface - the right call when a compliance letter matters more than data residency, and the wrong one when residency is the whole constraint.

Bedrock AgentCore, the Gemini Enterprise Agent Platform (formerly Vertex AI), and similar cloud offerings give you scale and low operational overhead, coupled to a vendor’s cloud. That coupling is precisely the point of the ownership question: convenient to start, harder to leave.

Hermes Agent (Nous Research, MIT-licensed, self-hosted) is the one I’d point a single person to. It keeps persistent memory across sessions and writes its own reusable skills against the open agentskills.io standard, so it genuinely grows with you - a multi-platform gateway that runs on your own machine. Read it against my axes, though, and you can feel the design center: it’s built for a personal, single-user assistant, where Trinity is built for governed, multi-user production - different questions, and worth knowing which one you’re actually asking.

The takeaway

Every platform on this list will run your agents. That’s table stakes now, and it’s exactly why feature-column comparisons feel oddly interchangeable when you line them up. The decision that outlasts the demo is about ownership of the compounding asset - because an agent that learns on someone else’s platform is quietly compounding value into someone else’s moat.

That’s the lens I’d bring to any evaluation, and it’s the one I’ve written up in full, with the complete methodology and the platform-by-platform detail, in the canonical comparison of AI agent platforms to run agents in production. Start from your non-negotiable constraint - sovereignty, certification, coding focus, memory - and the shortlist gets short fast.

This article accompanies our new paper, “A Measurably Self-Improving Multi-Agent Forecasting System: Five Feedback Loops, No Retraining”: https://github.com/Abilityai/trinity/blob/88effe17c8a186efed887d140ca48a8dc82d26c4/research/closed-loop-forecasting/closed-loop-forecasting-2026-07-11.pdf, co-authored with Dmytro Filatov.
The fleet’s live record - every prediction, resolution, and Brier score - is public at https://brierhq.ai.
Prefer to watch? I walk through the whole system - the loops, the hidden-state store, and the live dashboard - in a 11-minute video.

---

TL;DR

- We deployed a system of seven LLM forecasting agents, a meta-analyst, and a knowledge-base agent that improves through five closed feedback loops - with zero retraining. It has been running continuously since February 2026: 96,161 logged predictions, 73,552 resolved against real-world outcomes.

- The headline finding: 93.5-99.7% of prediction error is noise (inconsistency), not bias. The standard LLM calibration toolkit targets bias - the minority component. Measure the decomposition before you tune anything.

- Feedback works where it is absorbed: three of five compared agents improved significantly on both Brier score and surprise rate. Two moved *against* their feedback. We report the split rather than hide it - absorption is the open problem.

- The system’s most unusual object is a store of 56,347 empirically scored hypotheses about the hidden state of the world. A hypothesis that stops winning is not deleted - it fires a regime-change alert.

- A fifth loop generates *what to forecast* through abduction from anomalies, because a scoring engine has a structural blind spot: it can only improve forecasts it was asked to make.

The Premise: What Can Be Measured Can Be Engineered

Forecasting has a public image as an unmeasurable art. It is the opposite. A probability judgment can be scored against the outcome it predicted (the Brier score - mean squared error between your stated probability and what happened, lower is better). The error can be decomposed into systematic bias and noise. And the central lesson of Tetlock’s superforecasting research is that forecasters who keep score improve.

That measurability is the entire premise of the paper: once forecasting error is measured and decomposed, each component becomes an engineering target. You can build feedback loops around a running system the same way you build them around a thermostat.

Most LLM forecasting deployments do not do this. A model is prompted, generates a probability, the result is logged. If the system improves at all, it improves through retraining - new weights, new evaluation, new deployment cycle. We built the opposite: a system where the improvement happens in the loops around the models, while the models stay frozen.

Here is what five months of production data taught us.

The System: Seven Forecasters, an Analyst, and a Librarian

The architecture has three agent types:

1. The Oracle fleet- seven LLM forecasting agents, each specialized in a domain: financial markets, AI and technology, geopolitics, energy, space and shipping, semiconductors, and biotech. Each generates 10-25 predictions every four hours and resolves its backlog against tiered web sources every two. Every prediction records a statement, a probability, a time horizon, and the full chain of reasoning.

2. Cleon, the meta-analyst - measures everything. Brier scores, calibration error, surprise rates, and a full noise decomposition per agent. It coaches each Oracle with calibration feedback, routes incoming questions to the agent with the best domain track record, and maintains the hidden-state hypothesis store.

3. Cornelius, the knowledge base - a Zettelkasten-style graph with semantic search. When a prediction fails badly, Cleon sends the failure to Cornelius, which extracts a durable lesson. Before any Oracle predicts anything, it queries Cornelius for relevant lessons first.

![The architecture: Cleon (meta-analyst), the seven-agent Oracle fleet, and Cornelius (knowledge base), connected by five feedback loops.]

The scale, as of the paper’s data snapshot: 96,161 logged predictions, 73,552 resolved outcomes, all binary, all scored. The whole fleet runs on [Trinity](https://github.com/Abilityai/trinity), our open-source runtime for autonomous agents, and an open-source template of the knowledge-base agent is [also public](https://github.com/Abilityai/cornelius).

The important design property: every feedback signal in the system is grounded in resolved external outcomes. Nothing improves because a model judged its own output to be good. This matters because the research literature has documented exactly how self-judged improvement loops fail - models reward-hack their own evaluations and resist correcting confident mistakes. Reality is the only referee that cannot be gamed.

The Finding That Should Change How You Tune LLM Agents

Here is the result I keep coming back to.

Take Kahneman’s error decomposition from Noise - the distinction between bias (systematic error in one direction: always too confident, always too optimistic) and noise (inconsistency: the same judge giving different answers to equivalent problems). Apply it to seven LLM forecasting agents across domain-by-horizon cells, over tens of thousands of predictions.

The result is consistent across every single agent: noise accounts for 93.5-99.7% of prediction error. Systematic bias never exceeds 6.5%.

![Error decomposition per agent: the systematic-bias share (blue) never exceeds 6.5%. Noise dominates in every single agent.]

Think about what that means. Nearly the entire LLM calibration toolkit - bucket recalibration, temperature scaling, confidence-elicitation tuning - targets the bias component. Applied to a noise-dominated system, it is aimed at, at most, a few percent of the problem.

This is not a quirk of our fleet. Independent benchmarking found the same phenomenon: the same model, asked the same forecasting question in different runs, can answer below 20% and above 80%. Roughly 60% of the variance comes from the final judgment step, not from evidence gathering.

So the system routes accordingly. We built a noise-dominance gate into the calibration loop:

- If noise dominates (currently: every agent) - the feedback is process consistency guidance: structured reasoning templates for high-noise question types, temporal drift alerts, and per-category adjustments instead of global nudges.

- Only when bias is a meaningful share does the feedback become the classic “adjust your probabilities in bucket X” correction.

The per-category point deserves emphasis. Because bias direction varies by domain and horizon, telling an agent “you’re overconfident” globally helps some question types while actively hurting others. Noise-aware feedback has to be targeted or it becomes noise itself.

The practical takeaway for anyone building LLM agent systems: measure the error decomposition first, then pick the intervention. The default toolkit assumes a bias problem you probably do not have.

The Five Loops

Each loop targets a distinct source of prediction error. Four are corrective - they run downstream of a prediction. One is generative, and I will get to it last, because it exists to fix a blind spot the other four cannot see.

| # | Loop | Error source it corrects |

| 1 | Noise-adaptive calibration feedback | Inconsistency and conditional bias |

| 2 | Surprise-driven insight extraction | Wrong mental models |

| 3 | Knowledge retrieval before prediction | Repeated cross-agent mistakes |

| 4 | Hidden-state hypothesis injection | Ungrounded assumptions about the present |

| 0 | Abductive synthesis | Questions nobody thought to ask |

Loop 1 is the noise-gated coaching described above.

Loop 2 fires when a prediction fails hard - when the gap between stated probability and outcome exceeds 0.40. Cleon sends the full context to Cornelius with a structured prompt that asks what assumption about the world failed, not which agent was wrong. A real example from the store: “OPEC+ can execute emergency output changes faster than the assumed 3-month lead time. The model’s supply-side response lag is systematically wrong.”

Loop 3 closes the circuit: before generating any probability, every Oracle queries Cornelius - “what do we know about this topic?” - and reads the distilled lessons from every prior failure across the whole fleet. One agent’s mistake about OPEC+ supply response becomes another agent’s safeguard on LNG shipping routes. The fleet learns collectively without sharing any state directly - the knowledge base is the medium.

Loop 4 is the strangest and, I think, the most valuable. Every probabilistic prediction is implicitly a bet on the current, unobservable state of reality. “Oil exceeds $100 in Q3 at 75%” silently assumes specific hidden conditions hold right now. So the system makes those assumptions explicit: after each resolution, it extracts the hidden-state hypotheses the reasoning relied on, deduplicates them into a store, and scores each one by the win rate of the predictions that used it. Before each new prediction, the relevant hypotheses - with their empirical track records - are injected into the Oracle’s context.

![The assumption store: status distribution, uses per hypothesis, and the win-rate gradient across lifecycle tiers.]

The store currently holds 56,347 hypotheses, and the validation is clean: hypotheses graduate through a lifecycle (emerging, strong, fading) based on recency-weighted win rates, and the win-rate gradient across tiers is monotone - strong 75.0% > emerging 68.2% > fading 59.0%. The scoring separates reliable world-context from stale world-context.

The best part is what happens when a hypothesis stops working. It is not deleted. A hypothesis that scored well for months and then starts losing fires a regime-change alert - the world probably changed, and that is actionable information in itself. A live example: “FDA reviewers have not identified any new safety signal in the IV nimodipine formulation” was reliable context for 117 predictions, then its recent confidence collapsed to 0.22. That collapse is the signal.

The Evidence: An Honestly Split Fleet

Does the feedback actually change behavior? The honest answer is: yes, where it is absorbed - and absorption is not guaranteed.

Since calibration feedback began in March 2026, three of five compared agents improved on both Brier score and surprise rate, with 95% bootstrap confidence intervals excluding zero on both metrics. Fleet-wide surprise dropped from 48.7% to 44.4%.

The cleanest causal chain is the biotech agent. Told repeatedly it was about 11 points underconfident, it raised its mean confidence by 7.7 points - with every biotech sub-domain moving in the instructed direction - cut its surprise rate from 53.4% to 41.5%, and improved its Brier score while making the shift. Feedback direction matched on every axis. That is the archetype the loop is designed to produce.

![Loop 1 before/after evidence with 95% bootstrap confidence intervals. Three agents improve on both metrics; two regress against their feedback.]

And then there is the counter-signal: two agents - markets and semiconductors - were told to raise confidence and did the opposite, and their surprise rates rose. The loop delivers advisory text at prediction time; nothing forces a model to follow it. Two of seven agents moving against direction-explicit advice is a limitation we report in the paper rather than bury, and it defines the next engineering problem: when should a global advisory become a per-domain correction, and when does an agent need a different mechanism entirely?

We also refused to grade our own homework on accuracy. Raw Brier scores flatter agents on lopsided question mixes, so we anchored everything against baselines computed on *exactly the same questions*: the fleet scores Brier 0.194 against 0.250 for always-guessing-50% and 0.217 for an oracle-advantaged base-rate forecaster - a skill score of +0.105 with a tight confidence interval. Five of seven agents beat their own base rate; one ties it; one falls below it despite a mid-fleet raw Brier, which is precisely why baseline anchoring matters. For context (different question sets, so not head-to-head): superforecasters score 0.096, human crowds 0.149-0.19, and current LLM systems cluster around 0.12-0.21. The fleet sits in the LLM band - real but bounded skill, stated plainly.

![Left: each agent against the base-rate baseline on its own questions. Right: published Brier scores from the literature - context, not a head-to-head comparison.]

Loop 0: The Questions Nobody Asked

The four corrective loops share a structural blind spot that took us months to see: they can only improve forecasts the system was asked to make.

Every hypothesis in the store was extracted from predictions the Oracles already made. The store is a mirror of what we chose to ask - not a searchlight on the world. A regime shift nobody is forecasting never enters the system at all. We confirmed this with a blind test: we queried the store for the four pillars of a circulating de-dollarization thesis - central banks quietly rotating into gold and building parallel settlement rails - and got back nothing but unrelated noise. The system had no opinion, because no prediction had ever pointed there.

The missing operation is abduction - inferring the non-obvious present state that would explain a cluster of surface anomalies, then deriving what else would be true if it held.

So Loop 0 runs upstream of everything: scan for cross-domain anomalies the current model does not explain, generate candidate hidden-state theses (each paired with its competing “official” explanation), derive at least three falsifiable consequences per thesis - “if this is true, we would also see X, Y, Z by date H” - and then commission those consequences as real predictions through the same resolution machinery as everything else. A thesis becomes a bundle of falsifiable forecasts with a scoreboard.

This is the part of the system I find philosophically satisfying. A human macro commentator produces bold, non-consensus theses but never has to settle them - no calibration, no scorecard, and the wrong ones fade from memory. Loop 0 generates theses just as bold, then kills the wrong ones with data in weeks.

Status, honestly reported: the loop is newly activated. Three theses are live - on the Hormuz shipping regime, US economic-security policy, and rates-driven energy tightness - with all ten derived consequences commissioned as real predictions. Zero have resolved; the earliest horizons land in September 2026. As of the paper, the generative loop is an architectural claim with its first report card pending. We publish the resolution dates so anyone can check.

What This Means If You Are Building Agent Systems

Four principles fall out of this work, and none of them are specific to forecasting:

1. Decompose error before tuning anything. If your agents’ error is 95% inconsistency, calibration nudges and prompt-level bias fixes are aimed at the wrong component. Measure first; the intervention follows from the decomposition, not from the default toolkit.

2. Ground every feedback signal in external outcomes. Self-judged improvement loops reward-hack. Resolved reality - a prediction that came true or did not, a test that passed or failed, a customer who converted or churned - is the only feedback source that cannot be flattered.

3. Expect absorption failure and measure it. Delivering feedback is not the same as changing behavior. Two of our seven agents moved against explicit advice. If you do not measure before/after per agent, you will assume your feedback works everywhere it is merely delivered.

4. Correction and generation are different layers. A system that only corrects what it was asked will be precisely wrong about the questions that matter most - the ones nobody asked. Budget for the generative layer, and force its output through the same scoring machinery as everything else.

The deeper pattern: the improvement lives in the system around the models, not in the weights. Model capability was held fixed for the entire five months - every gain came from measurement, routing, memory, and feedback structure. When the next model generation arrives, the loops transfer; the architecture is the asset.

The paper reports all of it - the architecture, the noise decomposition, the split-fleet evidence with confidence intervals, the hypothesis-store validation, and eight limitations stated without cosmetics (https://github.com/Abilityai/trinity/blob/main/research/closed-loop-forecasting/closed-loop-forecasting-2026-07-11.pdf), or watch the 10-minute walkthrough.

If you are building feedback loops around LLM systems - or have hit the absorption problem yourself - I want to hear about it. Reply or leave a comment: what does your system do when an agent ignores its own coaching?

Further Reading

- Philip Tetlock and Dan Gardner, *Superforecasting* - the keep-score-and-improve tradition this system industrializes

- Daniel Kahneman, Olivier Sibony, and Cass Sunstein, *Noise* - the bias/noise decomposition applied throughout

- Halawi et al., “Approaching Human-Level Forecasting with Language Models” - the retrieval-reason-aggregate LLM forecasting baseline

- Karger et al., “ForecastBench” - the contamination-free benchmark where superforecasters still beat the best LLM

- FutureSearch, “Bench to the Future” - independent replication of noise dominance in LLM forecasting

Related concepts:

- [[Noise Dominates LLM-Agent Error So Calibration Tuning Targets the Minority Component]]

- [[Judgment quality is measurable through decision bias and noise]]

- [[Corrective Loops Only Improve What They Were Asked - Generation Is a Distinct Layer From Correction]]

- [[Fleet Learning Without State Sharing - One Agent’s Failure Becomes Every Agent’s Safeguard]]

- [[Good Agents Optimise the Process Not the Outcome - and Learn Only Where Feedback Generalises]]

- [[Decision-Making process needs to be optimised to improve non-verifiable judgments]]

The discipline got its name two weeks ago. The evidence behind it is twelve months old - and it changes where engineering effort should go. This guide synthesizes that evidence into a framework you can apply, then walks a production implementation to show the framework in practice.

Who this is for: anyone moving from prompting single tasks to running agents unattended - over hours, overnight, or on a schedule - who wants a platform that actually runs them. The worked example throughout is Trinity’s loop primitive.

What you’ll learn:

1. Why the agent loop is a solved problem, and where the real engineering lives instead.

2. The four failure modes that kill production loops - and the four-layer framework that bounds each one.

3. How to package a unit of work as a playbook that runs unsupervised.

4. A simple rule for choosing between a loop and a schedule.

5. How to run these loops unattended on Trinity - a worked example turning every principle into a concrete design decision in a production system.

TL;DR

• The agent loop is a solved problem. Every major platform converged on the same while-loop. A 100-line agent scores 74%+ on SWE-bench Verified. All meaningful engineering lives in the harness around the loop: termination, verification, context management, cost enforcement.

• State belongs in files and git, not the context window. The context window is compute, not storage. Fresh context per iteration beats accumulated conversation history for long-running work.

• Three hard stops are mandatory, not one. Max iterations, wall-clock timeout, and a hard cost ceiling. The canonical incident: four agents in an unguarded loop ran for 11 days and cost $47,000 - with budget alerts configured, but no enforcement.

• Verification displaced generation as the bottleneck. A production system merged 1,094 agent-written PRs with zero regressions because its tests were tamper-resistant. Code production is now a commodity; proving the code works is the constraint.

• The unit of work is an intent contract. Package work as a playbook - read state, do one unit, verify against evidence, write state, exit clean when idle. A playbook running at 3am cannot ask you anything; everything you would have said must already be in it. Built that way, the same playbook mounts on chat, schedule, or loop unchanged.

• Schedules drain flows; loops drain stocks. A selection rule for when to reach for each - plus a worked example: how Trinity’s loop primitive implements the findings above, and the design decisions behind it.

• You can run this today, not just read about it. Trinity ships the loop as a server-side primitive - hand an agent a goal and bounds (max_runs, a timeout, an optional [[DONE]] stop signal), disconnect, and it iterates unattended, every run bounded by its gates and visible in a Loops tab. Close the laptop; at 3am the loop is still running.

Introduction

Boris Cherny created Claude Code. As of June 2026, he has not written a line of code by hand in eight months. More than 80% of Anthropic’s merged code is authored by Claude, and Cherny describes his job in five words: “my job is to write loops.”

That sentence compresses a real shift in how agentic systems get built. The unit of work is no longer the prompt - a single instruction you supervise to completion. The unit of work is the loop: a system that re-invokes an agent against a goal, feeds it a verification signal, and runs until a condition is met or a guardrail fires.

The term “loop engineering” is new. Peter Steinberger and Addy Osmani named it in posts on June 7-8, 2026. But the practice is at least a year old, and over that year practitioners and researchers have accumulated something rare in AI engineering: convergent, quantified evidence about what works and what fails.

The rest of this guide is built in four moves. First we establish why the loop itself stopped being the interesting part. Then we catalog how loops fail, because the framework is a direct answer to those failures. Then we assemble the framework - four layers - and the unit of work it drives. Finally we ground all of it in a production implementation, the loop primitive we built into Trinity, to show how each principle becomes a concrete design decision.

The practical upshot, up front: everything in this guide is runnable today, and you don’t have to build the harness from scratch to use it.

Trinity

ships the loop as a server-side primitive - you hand an agent a goal template and bounds (max_runs, a timeout, an optional [[DONE]] stop signal), disconnect, and it iterates unattended, bounded by the gates, every run visible in a Loops tab. Close the laptop; at 3am the loop is still draining its bucket. The worked example below walks the exact design - so as you read each principle, picture the loop you would actually launch.

The Loop Is Solved. The Harness Is the Engineering.

Strip away the branding and every major agent platform - Claude Code SDK, OpenAI Agents SDK, LangGraph, OpenHands, Cursor’s background agents - runs the same control flow: call the model, inspect the output for tool calls, execute them, append results, repeat until the model produces a final answer or a limit trips.

The existence proof that this loop carries no secret sauce is mini-swe-agent: a 100-line Python implementation from the Princeton SWE-agent team, with a single bash tool and no tool-calling interface at all, that scores 74%+ on SWE-bench Verified - matching frameworks orders of magnitude more complex. Meta, NVIDIA, IBM, and Stanford use it as a research baseline.

Steve Kinney’s March 2026 analysis states the implication directly:

“The loop is a solved problem. The engineering around the loop - context management, safety controls, graceful degradation, cost containment - is where all the interesting decisions live.” - Steve Kinney, The Anatomy of an Agent Loop

Addy Osmani’s formulation of the same point: “A decent model with a great harness beats a great model with a bad harness.” A 54-page academic survey (Zhou et al., arXiv:2604.08224, April 2026) formalizes it - agent capability increasingly depends on externalized infrastructure (memory, skills, verification, coordination), not on stronger weights.

Here’s what this means in practice: if you are choosing between investing a week in framework sophistication and a week in your test suite, stop conditions, and state files, the evidence says the second week pays more.

The Founding Pattern: Naive Persistence, Durable State

The practice that became loop engineering started with something deliberately dumb. In July 2025, Geoffrey Huntley published the Ralph Wiggum Loop - named after the Simpsons character who rams his head into things while announcing “I’m helping!”:

while :; do cat

PROMPT.md

| claude-code ; done

Restart the agent forever against a prompt file describing the desired end state. That’s the whole loop. The results were not dumb: an engineer delivered a $50,000 contract MVP for $297 in API costs, and the loop built a production-grade programming language (CURSED, with LLVM compilation) in a syntax that appeared nowhere in any training data.

The mechanism that makes it work is the part most people miss. Each iteration starts with a fresh context window. No accumulated conversation, no pile of prior failures. The agent orients itself the same way a new engineer would: by reading the filesystem and the git log. State lives in files and version control - durable, inspectable, recoverable - never in the model’s context.

This inverts the intuition that more context means better performance. In long sessions, failed attempts accumulate until the original specification gets buried or dropped - the agent stays confident while drifting off-goal. This matches something I observed in my own agents long before I had the vocabulary for it: [[Agents lose big picture in long sessions like humans lose forest for trees]]. The Ralph pattern dissolves the problem structurally. There is no long session. There is only the current state of the world and a goal.

The principle generalizes beyond coding loops, and it is the same one behind [[The Folder Paradigm - agents own directories as operational memory]]: the context window is compute, not storage. Treat it like CPU registers - transient working space - and put everything durable on disk. Anthropic’s own guidance for long-running agents (November 2025) lands on the identical architecture: an initializer agent writes progress files and a feature list, then each subsequent session reads state, does one unit of work, commits, and exits clean. One detail from that guide worth stealing: structured state files survive better as JSON than as Markdown, because models are less prone to “helpfully” rewriting them.

How Loops Fail: The Doom Loop, Context Rot, and a $47,000 Invoice

Production reports from 2025-2026 converge on a small taxonomy of failure modes. Two deserve special attention because they are structural, not random.

The doom loop. A tool call returns full state - an entire file, a complete DOM, a full error log. That output re-enters the context, making the same tool call look freshly relevant, so the agent issues it again. Each repetition adds bulk to the context, accelerating the rot that caused the repetition. This is a compounding failure with a specific signature: identical (tool name, arguments) pairs across consecutive turns. It frequently stays under the iteration limit, which is why a turn cap alone won’t catch it.

Slow structural degradation. SlopCodeBench (Orlanski et al., arXiv:2603.24755, March 2026) measured how coding agents degrade over long-horizon iterative tasks: no agent fully solved any of 36 problems end-to-end (best checkpoint pass rate: 14.8%), 77% of trajectories showed structural erosion, and agent code came out 2.3x more verbose than human code. The most uncomfortable finding: explicit quality guidance in the prompt reduced initial problems by 33% but did not slow the rate of degradation across iterations. You cannot prompt your way out of the erosion arc. You can only bound it - with fresh context, verification gates, and short task units.

And then there is the runaway loop, the most common failure and the most expensive. The canonical incident: four agents in an unguarded loop ran for 11 days and produced a $47,000 bill. The detail that matters is that budget alerts were configured. Alerts fire after spend occurs. Only a hard ceiling - enforced before the next API call - is budget control. This is not a tooling subtlety; it is the difference between a thermostat and a fire alarm. (It is also a textbook ergodicity failure in the sense of [[Ergodicity and Impermanence - Survival Over Optimization]]: averages are irrelevant when one path hits the absorbing barrier. Survival constraints come before optimization.)

The Framework: Four Layers of a Production Loop

Each failure mode from the previous section has a structural answer, and together those answers form a framework. Synthesizing the consensus across practitioner posts, platform documentation, and the 2026 papers, a production-grade loop has four layers - read the table as failure-on-the-right, defense-on-the-left:

The four layers of a production loop

1. Termination

• Components: max iterations (15-25) + wall-clock timeout + hard cost ceiling

• Prevents: runaway loops - the $47k incident

2. Progress detection

• Components: fingerprint identical (tool, arguments) tuples; exit on no state change

• Prevents: doom loops running under the turn cap

3. Verification

• Components: deterministic tests > end-to-end automation > LLM judge > human gate > self-evaluation

• Prevents: confident garbage, reward hacking

4. Durable state

• Components: files + git as memory; fresh or compacted context per iteration

• Prevents: context rot, unrecoverable failures

First, the feedback signal hierarchy is strict. Deterministic signals (test suites, type checkers, linters) beat probabilistic ones. Model self-evaluation sits at the bottom: models systematically skew positive when grading their own long-horizon output. If your loop’s exit condition is “keep going until the output is good” judged by the same model that produced the output, your loop will exit on the first pass, satisfied with mediocrity. Separate the generator from the evaluator.

Second, verification infrastructure must be tamper-resistant. The Kitchen Loop paper (Roy et al., arXiv:2603.25697, March 2026) ran 285+ loop iterations across two production systems and merged 1,094 pull requests with zero regressions. The authors attribute the record to one architectural constraint: the code-writing agent could not modify the tests that judged its output. Their conclusion reframes the whole field: “Code production is now a commodity; the bottleneck is knowing what to build and proving it works.” The generation bottleneck is gone. The verification bottleneck is what loop engineering actually manages.

Worked Example: The Trinity Loop Primitive

Theory is cheap, so let me show how these findings translate into a real implementation. Trinity - our agent orchestration platform - ships a server-side loop primitive: you hand an agent a message template and bounds, get back a loop_id, and disconnect. The backend runs every iteration in order through the standard execution pipeline. It is exposed in the UI (a Loops tab per agent), via REST, via MCP tools (run_agent_loop, get_loop_status, stop_loop), and through a /trinity:loop skill in the plugin marketplace that gives any local Claude Code agent a durable remote counterpart to its built-in /loop.

Why server-side at all? Because a laptop loop has a ceiling. Claude Code’s built-in /loop runs iterations on your machine, in your session, and that imposes three structural limits. It is fragile: close the lid and the loop dies; and in my use it iterates inside the same accumulating session rather than a fresh context per run, so a long loop inherits the context-rot arc that fresh-context architectures exist to prevent. It is bounded: want twenty loops running, and your machine is the bottleneck. And it misses the point: the whole point of a loop is that it runs when you are not watching. The working model that resolves this is a synchronized pair - I run a local copy and a remote copy of the same agent, kept identical through git, and the /trinity:loop skill lets the local one hand the iteration off to its remote counterpart and disconnect. Close the laptop; at 3am the loop is still draining its bucket, bounded by its gates.

Walking through its design decisions against the framework above:

Termination: the cap always wins. Every Trinity loop requires max_runs (1-100, hard ceiling) and carries a timeout_per_run (10-7200 seconds) - layers 1’s iteration cap and wall clock. There are exactly two modes, and they mirror the field’s consensus exit paths. Fixed mode runs exactly N iterations. Until mode adds a stop_signal - a substring like [[DONE]] that the agent emits when the condition is met, ending the loop early. The critical design rule: the stop signal is best-effort, the cap is guaranteed. An agent that never emits the sentinel cannot loop forever; max_runs always wins. This is the “done tool” forcing pattern and the safety limit, composed.

Verification: tie the sentinel to evidence, not self-assessment. The skill documentation encodes the self-evaluation finding directly: the until-condition must be rewritten into something checkable before the loop fires. Not “loop until the report is good” - that exits on iteration one, because models grade their own work generously. Instead: “run the test suite; if every test passes, paste the passing output and end your reply with [[DONE]].” The sentinel rides on verifiable evidence - test output, an HTTP 200, zero remaining TODO markers. Subjective conditions get rewritten or get a low cap.

Durable state: the workspace persists, the loop context does not. Trinity loops support chaining - {{previous_response}} injects the prior iteration’s output into the next prompt. But it is deliberately lossy: the trailing 2,000 characters, a hint rather than an artifact. When iterations build something real, the instruction is the Ralph instruction: keep the artifact in a file in the agent’s workspace and re-read it each run (”refine the draft in

report.md

- read it first, improve it, write it back”). The remote agent’s filesystem persists across iterations; each iteration’s context does not. That is [[The Context Window is Compute Not Storage]] as an API contract.

Progress detection: watch for stalls the cap can’t catch. A loop producing near-identical responses each iteration - same failure, same output, no state change - is burning budget under the limit. Trinity’s status view surfaces per-run summaries precisely so this signature is visible, and the skill’s observe phase flags it and recommends a stop. This is the doom-loop fingerprinting insight applied at the orchestration layer.

Operational details that the research says matter: stops are cooperative (the in-flight iteration finishes; no corrupted half-state), failures are fail-fast (a failed iteration ends the loop rather than compounding on a broken foundation - the error-compounding defense; more on when that default is right below), and every iteration is a first-class execution row tagged with its loop_id, visible in the timeline with its own cost and duration. Loops appear as their own analytics category. You cannot manage the economics of a loop you cannot see. One more property worth noting: loops compose with the permission layer. A looped agent can message any agent it has been granted access to, so a single bounded loop can drive a multi-agent workflow - the loop supplies the budget and the termination gates while the agent fans the work out.

None of these decisions came from the June 2026 posts - the primitive predates the term. That is rather the point. The constraints of running unattended agent iterations in production push every serious implementation toward the same shape: bounded, evidence-terminated, file-stated, observable. The same convergence shows up across Claude Code, OpenAI’s SDK, LangGraph, and OpenHands. When independent builders keep arriving at identical architecture, you are looking at engineering reality, not fashion.

The Unit of Work: A Playbook Is an Intent Contract

The loop is the driver. What it drives deserves equal design attention, and it has a fixed anatomy. The unit of work - I call it a playbook: a packaged, reusable procedure an agent executes - consists of four steps:

1. Read state. Orient from files and git - never from a stale context window.

2. Do one unit. The smallest meaningful increment of work.

3. Verify. Produce evidence the work is done - tests, checks, a sign-off checklist. Never self-assessment. Aviation solved this problem decades ago: pilots do not “feel confident about the flaps,” they sign off items on a checklist. Give agents the same instrument.

4. Write state. Commit results to files and git, report, exit.

One rule sits on top of the four steps: nothing to do? Exit clean. The box must be cheap when idle and safe to re-run.

The reason this anatomy is non-negotiable for autonomous work is what I call the intent contract. A playbook for a supervised chat can ask you a question mid-flight. A playbook running at 3am cannot. Everything you would have said in the conversation must already be in the playbook - the decision criteria, the edge-case handling, the definition of done. Writing one is less like prompting and more like writing the brief you would hand a contractor before leaving for vacation. And the verification step inherits the evidence rule from layer 3: not “loop until the report is good” - that exits on pass one - but “run the tests, paste the passing output, then say done.”

Package the work this way and the same box mounts on three drivers without modification: chat (on demand, supervised), schedule (cron tick, runs forever), loop (burst of N runs, terminates). Chat needs no selection rule - you are sitting right there. Between the other two, the choice deserves its own section.

When to Loop and When to Schedule

A loop is not the only driver for autonomous work - Trinity, like most platforms, also offers cron schedules. Practitioners need a selection rule, and the one I use is this: schedules drain flows; loops drain stocks.

A schedule is the right mount when work arrives continuously and unpredictably - new notes appear, external signals shift, items trickle into a queue. Because you don’t know when work shows up, the scheduled playbook pays a design tax: it must be self-gating (check for work, exit clean when there is none), idempotent, and near-free when idle. Pay that tax and it runs forever. Cornelius, the agent that maintains my knowledge base, runs this way: a daily incubation tick that advances open research questions one analytical move at a time, persisting its reasoning state in files - and its git log is full of “clean exit, no active topics” entries. That’s the tax working.

A loop is the right mount when the work is a finite stock that exists right now: 20 unprocessed book chapters, 6 recommended wikilinks at the end of an analysis report, one question you want hammered for 8 iterations tonight. The loop skips the self-gating tax entirely, because the gate was you - a human looked at a pile and decided it deserved a burst. That is also why the loop launch needs no automation safeguards around judgment: the launch is the judgment act. What it needs instead is termination engineering, which is where the four layers above take over.

Inside a loop, iterations take one of two shapes, and naming them sharpens the design. Independent runs over stable state: each run takes the next item from the bucket - 20 chapters, one per run. The runs share a workspace but not a dependency. Converging runs over incremental state: each run updates the state the next run reads - the draft improves until [[DONE]]. The shape settles a question every loop implementation has to answer: what happens when a run fails halfway? Converging loops must stop - the next iteration would build on a broken foundation, and error compounding does the rest. Independent loops may legitimately log the failure and keep draining - the failed item gets picked up on a later pass. The failure policy follows the iteration shape. (Trinity currently ships the conservative default: any failed run ends the loop.)

The corollary that makes this practical: the unit of work is invariant; only the driver changes. A playbook built to the standard above - small unit, file-based state, evidence-gated verification, clean exit - is already a valid loop body. Cornelius’s incubation playbook proves it: cron-mounted, it thinks one move per day; loop-mounted (run one incubation move on topic X, 8 times), it converges overnight. Same unit, same state files, different driver. So the design rule is: always build the unit to the schedule standard, then choose the mounting per situation. Other stock-shaped loops from the same agent: ingest the next unprocessed chapter, [[DONE]] when the changelog shows none remain; apply the next recommended wikilink from a connection report, [[DONE]] when the list is empty.

(One case outside the stock/flow frame: bounded polling - “check the deploy every 2 minutes until healthy, max 30.” A loop waiting on an external state change with a deadline. It completes the taxonomy.)

How to Apply This

If you take three things from this guide, take these:

1. Reallocate your effort. The loop control flow deserves an afternoon. The harness - tests the agent cannot game, state files, stop conditions, cost ceilings - deserves the rest of the quarter. The mini-swe-agent result is the permission slip: simplicity with rigor beats complexity.

2. Adopt the survival checklist before scaling anything. Iteration cap, wall-clock timeout, hard cost ceiling, stall detection, evidence-based exit conditions. Each guards a failure mode the others miss. The $47,000 incident happened with monitoring in place - what was missing was enforcement.

3. Build verification before building loops. A loop amplifies whatever signal it is fed. With tamper-resistant tests, that amplification produced 1,094 regression-free PRs. With self-evaluation, it produces confident slop at scale. The bottleneck moved; move your investment with it.

The deeper shift is the one Cherny’s quote points at. Prompting is piecework - your output is bounded by your attention. A loop is a machine that converts a goal specification plus a verification signal into iterations, unattended. Engineers who internalize this stop asking “what should I tell the agent to do?” and start asking “what loop, fed what signal, bounded by what stops, converges on this outcome?” That is a different job. It is also, increasingly, the job.

References & Further Reading

• Geoffrey Huntley, “Ralph Wiggum as a ‘software engineer’” (July 2025) -

  https://ghuntley.com/ralph/

• Addy Osmani, “Loop Engineering” (June 2026) -

  https://addyosmani.com/blog/loop-engineering/

  and “Agent Harness Engineering” (April 2026)

• Steve Kinney, “The Anatomy of an Agent Loop” (March 2026) -

  https://stevekinney.com/writing/agent-loops

• Anthropic Engineering, “Effective Harnesses for Long-Running Agents” (November 2025) and “Effective Context Engineering for AI Agents” (September 2025)

• Roy et al., “The Kitchen Loop” (arXiv:2603.25697, March 2026)

• Orlanski et al., “SlopCodeBench” (arXiv:2603.24755, March 2026)

• Zhou et al., “Externalization in LLM Agents” (arXiv:2604.08224, April 2026)

• mini-swe-agent -

  https://github.com/SWE-agent/mini-swe-agent/

• Dex Horthy (HumanLayer), “A Brief History of Ralph” (January 2026)

• “The $47,000 Agent Loop” -

  dev.to/waxell

  (2025)

• Boris Cherny via Office Chai, “I Now Just Write Loops” (June 2026)

• Trinity docs, “Sequential Agent Loops” - Trinity user documentation, automation/agent-loops

• Eugene Vyborov, “Loop Engineering for Autonomous Agents with Trinity” - Trinity workshop, June 11, 2026 (live demos: incubation loop convergence, multi-agent loop via permission layer, local-to-remote loop handoff)

TL;DR

• The incubation loop applies rotating analytical moves (Bayesian updates, steelmanning, ACH, cross-domain bridging) to your open questions on a schedule - the architectural equivalent of what the brain does during hypnagogic sleep

• The domain watch scans your knowledge base for new notes that match declared intellectual interests, probes external signals, and auto-activates high-resonance findings into the thinking queue

• The insight interview searches your existing notes before asking a single question - it grounds Socratic dialogue in what you already think, not in what sounds generically useful

• All three require persistent autonomous operation, which is why Trinity is now the recommended deployment model for Cornelius

• YouTube transcript extraction closes the last obvious gap in the capture layer

Introduction

Most knowledge management systems are reactive. You query them. They respond. The insights you stored six months ago sit dormant until you remember to look. The connections between your notes on distributed systems and your notes on organizational behavior exist only as potential - unrealized until someone thinks to ask.

This is the correct design for a retrieval engine. It is the wrong design for a thinking partner.

The distinction matters because the most valuable cognitive work - the kind that produces genuine insight rather than retrieved information - happens at the edges. At the edges of sleep. At the edges of disciplines. At the edges of certainty, where a question has been sitting long enough that it has started to generate its own gravitational pull.

Cornelius v05.26 is an attempt to build those edges into the architecture.

What the Research Shows About “Edge” Cognition

Before describing the new capabilities, it is worth understanding the neuroscience that motivated the design.

A 2024 study published in Science Advances quantified something practitioners have suspected for centuries: the hypnagogic state - the liminal zone between wakefulness and sleep onset - produces an *83% insight solution rate* on creative problem-solving tasks. Full wakefulness produces 30%. Deep sleep produces only 14%.

This is counterintuitive. Deep sleep, the state most associated with rest and recovery, is the worst state for generating novel connections. The sweet spot is not rest or alertness - it is the threshold between them.

The mechanism is specific: during hypnagogic onset, logical monitoring from waking consciousness is still partially active, while associative loosening from early sleep onset enables distant semantic connections that executive control would normally filter out. Edison and Dali both independently discovered this and built nap techniques around it. The neuroscience now explains why their techniques worked.

There is a direct architectural parallel here. The hypnagogic state maps cleanly onto the problem with always-on reactive knowledge systems: they operate at full executive control. Every query is deliberate. Every response is filtered by what you consciously thought to ask. The hypnagogic insight rate of 83% - versus 30% for wakefulness - suggests there is significant cognitive value left on the table when you only query your knowledge at full attention.

A second research thread from a 2026 Physiological Reviews synthesis reinforces this point: Sleep as Biological Zettelkasten. The hippocampal-neocortical dialogue during sleep is structurally isomorphic with how a Zettelkasten builds knowledge. The hippocampus acts as an inbox (fast, episodic, temporary). The neocortex acts as permanent notes (slow-learning, abstract, long-term). Sleep replay (sharp-wave ripples) acts as the linking process - pushing episodic content toward neocortical integration. The implication: a knowledge system that only operates when queried is missing the offline consolidation pass that the brain runs automatically every night.

Cornelius v05.26 is the attempt to implement that offline pass.

The Incubation Loop: Rotating Analytical Moves on a Schedule

The incubation loop is the centerpiece of this release. It is an autonomous iterative thinking engine that processes your active open questions using a rotating set of analytical moves.

Each scheduled run applies one move from this set:

1. ACH audit - Analysis of Competing Hypotheses. Forces explicit enumeration of alternative explanations and maps which evidence supports or undermines each.

2. Bayesian update - Integrates new information into probability estimates. Does confidence shift with the latest available signal?

3. Steelman opposition - Builds the strongest possible case for the position you are leaning away from.

4. Cross-domain bridge - Searches for structural analogies in adjacent domains. What does the military say about this? What does evolutionary biology say?

5. Implication check - If the current leading hypothesis is correct, what follows? Are those implications consistent with other things you believe?

6. Assumption audit - What is the reasoning taking for granted? Which assumptions are load-bearing?

The rotation matters. Applying only Bayesian updates produces overconfident position drift. Applying only steelmanning produces paralysis. The rotation forces the question through multiple cognitive lenses without any single lens dominating.

Topics are managed in a registry at Brain/05-Meta/Thinking/THINKING-REGISTRY.md. Each topic gets its own reasoning file where the loop persists state across runs. Convergence is auto-detected: when the same hypothesis leads three or more consecutive runs with less than 5% confidence delta, the topic is flagged for manual crystallization via /synthesize-insights.

The crucial design principle: the loop runs on a schedule regardless of whether you are present. It is not waiting for your query. It is doing the work between your sessions, the way sleep does consolidation between learning sessions.

Domain Watch: A Perception Layer for Your Knowledge Base

The incubation loop processes questions you have already framed. Domain watch handles something earlier in the pipeline: noticing when new signal has arrived that is worth framing into a question.

Here is the problem it solves. Every week, new notes arrive in your knowledge base - document insights, captured thoughts, extracted research. Some of those notes will resonate with questions you are actively thinking about. Most will sit in the graph, linked by semantic similarity, never surfaced as relevant to your current intellectual agenda.

Domain watch is a configured surveillance layer. You declare the domains you are watching (geopolitical stability indicators, agent memory architectures, attention economy mechanisms, whatever your actual intellectual priorities are). You specify what counts as a signal worth escalating. Domain watch then:

7. Scans new notes in the KB for pattern matches against your declared domains

8. Checks gap resonance - does this new note address something the incubation loop has been circling without enough evidence?

9. Probes external signals via web search for the domain

10. Auto-activates HIGH and MEDIUM resonance findings into the Thinking Registry

The result is a system that brings relevant information to your active questions, rather than requiring you to remember to search for it.

This is the difference between a retrieval engine and a perception layer. A retrieval engine answers queries. A perception layer monitors the environment and surfaces what matters. Cornelius previously had only the former. v05.26 adds the latter.

Insight Interview: Socratic Dialogue Grounded in What You Already Think

The insight interview addresses a subtler problem: how do you extract your own thinking from your knowledge base, rather than information?

Generic Socratic dialogue is easy to generate. Ask broad questions about a topic, get broad answers, surface nothing that was not already obvious. The reason standard brainstorming sessions often produce mediocre results is that the questions are not grounded in the specific state of the thinker’s existing beliefs.

The insight interview starts differently. Before asking a single question, it:

11. Searches the KB for what you have already written about the topic

12. Reads the relevant permanent notes, extracted insights, and document insights

13. Identifies where your existing thinking is strong, where it is thin, and where it contains productive tensions

Only then does it begin the dialogue - one question at a time, each question targeted at the specific gap or tension in your existing notes. The conversation transcript is saved at the end, and the best extractions are run through the insight extractor to create new permanent notes.

The underlying logic is from research on expertise elicitation: the most effective questions are not generic prompts but targeted probes that surface tacit knowledge the expert holds but has not articulated. The KB search step is what makes this possible. Without knowing what you have already thought, any question is a shot in the dark.

Trinity: Why Autonomy Requires a Different Deployment Model

These three capabilities share a common dependency: they require the agent to run when you are not there.

Running Cornelius locally - starting Claude Code when you want to work, closing it when you are done - is fine for interactive knowledge work. It is structurally incompatible with an agent that is supposed to process your thinking overnight, scan for signals while you are in meetings, and surface convergence findings the next morning.

This is why Trinity is now the recommended deployment model for Cornelius.

Trinity is an open-source platform for self-hosting autonomous agent fleets. Each agent runs in an isolated Docker container with cron scheduling, real-time monitoring, and agent-to-agent communication. The scheduled runs that power the incubation loop and domain watch are managed by Trinity’s distributed scheduler with lock guarantees - they run exactly once per schedule window, even in multi-instance deployments.

The default schedule is:

• 5am UTC daily - Rebuild FAISS index and bootstrap the Brain Dependency Graph

• 7am UTC daily - Run the incubation loop on all active topics

• Monday 6am UTC - Run the coherence sweep (structural health, staleness, lifecycle briefing)

What this produces in practice: you wake up to a knowledge base that has already done a pass of analytical work on your active questions. The incubation loop ran at 7am. Domain watch flagged two new notes as HIGH resonance with your geopolitics questions. The coherence sweep identified that three of your crystallizing notes are ready for synthesis into a framework.

The agent thought while you slept. The 83% hypnagogic insight rate is not achievable through deliberate querying. It requires letting go of executive control and trusting the offline process.

YouTube Transcript Extraction: Closing the Capture Gap

The remaining new capability is simpler in design but significant in practice: /get-youtube-transcript extracts the full text transcript from any YouTube video by URL or video ID.

This closes the last obvious gap in the Cornelius capture layer. Text (PDFs, web articles, books via the epub extractor) was already handled. Audio from voice conversations was already handled. Video has been a manual transcription problem until now.

The workflow is: watch a lecture, podcast, or interview, then run /get-youtube-transcript on the URL, then pipe the result into /extract-insights for processing into the KB. The full insight pipeline from video content now runs without manual intervention.

The Pattern: From Retrieval to Reasoning to Autonomy

Looking at the Cornelius changelog across versions, there is a consistent trajectory:

• v01.25: Retrieval. Ask questions, get answers from stored notes.

• v02.25 - v03.26: Memory architecture. SYNAPSE-inspired spreading activation, intent classification, Q-value learning. The system gets better at retrieval based on how you use it.

• v04.26: Dependency tracking. The Brain Dependency Graph adds direction, lifecycle, and staleness. The system understands which notes are authoritative and which are stale.

• v05.26: Autonomous reasoning. The system applies analytical moves to your questions, monitors for incoming signal, and surfaces gaps in your thinking - without being asked.

Each phase adds a layer that the previous phase made possible. You cannot do autonomous reasoning without a well-structured knowledge graph. You cannot have a well-structured graph without good retrieval. The architecture compounds.

The question this raises is: what does the next layer look like? A persistent institutional intelligence that does not merely store and surface knowledge but represents it, speaks from it, and builds on it across years of continuous operation. That requires the autonomous layer being added now.

How to Get the New Capabilities

*For existing Cornelius users:*

The new skills are in the public repo at github.com/Abilityai/cornelius. Pull the latest and the new skill files will be available immediately.

For autonomous operation, deploy to Trinity using the trinity plugin:

claude plugin add abilityai/abilities
# Then from inside Cornelius:
/trinity:connect
/trinity:onboard

Full plugin documentation at docs.ability.ai.

*For new users:*

Start with the QUICKSTART.md to get the local setup running, then use /trinity:onboard to enable the scheduled autonomous capabilities.

The incubation loop requires seeding topics in Brain/05-Meta/Thinking/THINKING-REGISTRY.md. Domain watch requires configuring your surveillance domains. Both are managed via the /manage-thinking-topics and /manage-watching-domains skills respectively.

Conclusion

The gap between a knowledge base and a thinking partner is not a retrieval problem. Better search does not close it. Semantic similarity scores do not close it. What closes it is offline processing - the kind of analytical work that happens between sessions, on a schedule, without requiring your presence.

The neuroscience is clear on this: the hypnagogic state, not alert wakefulness, produces the highest insight rate. The brain’s most productive reasoning happens at the edges of consciousness, not at the center of deliberate attention.

Cornelius v05.26 is built on this principle. The incubation loop runs at the edges - between your sessions, applying structured analytical moves to questions that have been sitting long enough to generate real friction. The domain watch monitors the periphery. The insight interview excavates what you already know but have not yet articulated.

The knowledge base is no longer waiting for your query. It is working while you are not there.

TL;DR

• Self-improving AI systems are uniquely vulnerable to gaming their own evaluations because they can read the evaluation code they’re being measured against

• The ICLR 2026 Workshop on Recursive Self-Improvement (110 papers) identified this as the field’s central unsolved problem

• The solution for commercial deployments isn’t better detection - it’s a different architecture: use real-world business outcomes as your evaluation signal, inject them immutably, and gaming the evaluation mechanism becomes structurally impossible

• This same architecture simultaneously resolves objective drift and Goodhart’s Law at the deployment layer

• The “is it really self-improvement?” debate is a philosophical distraction - if the metric moves, the system improved

Introduction

In April 2026, 110 research papers converged at the ICLR Workshop on Recursive Self-Improvement to address the same central challenge: self-improving AI systems have a dangerous tendency to get better at their benchmarks without getting better at the underlying task.

The Darwin Gödel Machine jumped from 20% to 50% on SWE-bench. SICA, SSR, and AlphaEvolve posted landmark results across algorithm optimization and coding benchmarks. The field celebrated genuine capability gains. But buried in the experimental sections was an uncomfortable pattern: in some runs, agents were learning SWE-bench-specific optimizations rather than general software engineering. The benchmark score improved. The capability it was designed to measure did not.

The workshop’s diagnosis: this is not an edge case to monitor. It is the dominant failure mode of self-improving systems, and no one has fully solved it.

Here’s what the research community - working at the frontier of recursive self-improvement architectures - has not said clearly: for most commercial deployments, this problem was never hard to solve. The solution requires one architectural decision, made at the beginning. If you make it, gaming the evaluation mechanism becomes structurally impossible - not merely unlikely.

Why Self-Improving Systems Game Their Evals

To understand the solution, you need to understand why the problem exists.

A standard AI system cannot read its own evaluation code. It produces outputs; humans (or automated systems) evaluate those outputs against criteria the model never sees. The evaluation signal arrives from outside the system’s context window.

Self-improving systems break this separation by design. To improve itself, an agent must be able to assess its own performance, identify what it did wrong, and modify its behavior or architecture accordingly. This requires access to evaluation artifacts - what the tests check for, what patterns previously succeeded, what the scoring criteria are.

The same access that enables genuine self-improvement also enables gaming. When a DGM-style agent can observe which test cases it currently fails, it can do one of two things: genuinely fix the underlying capability, or specifically optimize for those test cases. Both produce higher benchmark scores. Only one produces a better agent.

Research from the ICLR 2026 RSI Workshop describes the mechanism precisely: “the moment benchmark score becomes the optimization target, it ceases to measure the thing it was designed to measure.” This is Goodhart’s Law applied to AI self-improvement - a law that appears to be mathematically unavoidable rather than merely probable.

The proposed mitigations from the research field include blinded tests, randomized evaluation criteria, diff classifiers requiring human-readable change rationales, and live benchmark collection (the SWE-EVO approach, which uses continuously refreshed real issues to prevent memorization). Each of these reduces gaming pressure. None eliminates it architecturally.

The Architecture That Makes Gaming Impossible

The reason these mitigations feel insufficient is that they treat eval gaming as a detection challenge. The right framing is architectural.

Gaming requires a lever. An agent can only game an evaluation if it has a mechanism to influence the evaluation signal - either by directly manipulating the evaluation code, or by learning to pattern-match against observable evaluation criteria.

Remove the lever, and gaming becomes impossible by construction.

Here is the architecture:

The External Anchor Principle:

1. Define success using real-world business outcomes - conversion rate, customer retention, tickets resolved per hour, revenue generated

2. Inject the current performance data into the agent’s context on every execution cycle as read-only input

3. Make the metric and target immutable from the agent’s perspective - it can see them but cannot modify them

4. Leave everything else fully modifiable: the agent’s behavior, its tools, its self-reflection logic, even its improvement mechanism

In this architecture, the evaluation signal is the external world. An agent that fakes improvement produces a detectable gap: its internal metrics improve while the business outcome doesn’t move. There is no lever to game the evaluation mechanism - the eval is the world, not a proxy for it.

An agent with sufficient capability and reach could in principle try to manipulate the anchor metric itself - generating low-quality conversions, optimizing for misleading signals. But that is a different class of problem (agent capability and containment) from eval gaming. The architecture eliminates gaming of the measurement system; the agent still has to actually move the real-world number.

The research field is searching for better proxies and better detection methods for gaming those proxies. The External Anchor Principle says: stop using proxies.

Three Problems Solved Simultaneously

The External Anchor architecture solves three distinct failure modes with a single structural decision.

1. Eval Gaming

Covered above. Without access to the evaluation mechanism, gaming collapses. The agent can optimize all it wants - it only gets rewarded when real-world outcomes improve.

2. Objective Drift

A second failure mode in recursive self-improving systems is objective drift: as an agent modifies itself over many cycles, its effective objectives can gradually shift from the original intent. Research on convergent instrumental goals predicts that sufficiently capable self-modifying systems tend to develop instrumental subgoals around resource acquisition and self-continuity, regardless of the original terminal objective.

The External Anchor prevents objective drift at the structural level. The terminal objective - expressed as the external business metric - is injected at runtime and cannot be rewritten by the agent. The agent can rewrite everything else. It cannot rewrite what success means.

This is not a trust-based safeguard; it is an architectural one. The agent is not constrained from drifting by rules or training. It is constrained by the fact that the metric it cannot modify is the only thing that determines whether its changes were improvements.

3. Goodhart’s Law

Goodhart’s Law holds that when a measure becomes a target, it ceases to be a good measure. This is the theoretical foundation of both eval gaming and objective drift - in each case, an internal proxy substitutes for the thing it was meant to measure, and optimization pressure corrupts the proxy.

The External Anchor Principle does not break Goodhart’s Law - it moves the Goodhartian corruption as far downstream as possible. Real-world conversion rates and revenue figures can still be optimized in ways that undermine deeper value: low-quality signups, short-term retention at the cost of long-term trust, high ticket-close rates achieved through deflection rather than resolution. But you have moved from “proxy corrupts in ways invisible to the system” to “proxy corrupts in ways visible to the humans running the business.” That is a meaningful structural improvement even if it is not a complete solution.

The lever still exists; it is just further downstream. Choosing the right anchor metric - one genuinely aligned with business value - is real work that happens before deployment, not a problem the architecture automatically resolves.

This same principle operates across multiple independent fields. Organizational designers separate welfare metrics from payouts to prevent gaming. Reinforcement learning researchers use verifiable rewards (RLVR) to bypass the reward model entirely. The ICLR 2026 safety track recommends separate evaluation environments kept strictly isolated from improvement infrastructure. Five independent fields have converged on the same structural insight: the evaluating reference must be external to the optimizing system.

What the Harness Actually Is

This framing points to a more general principle about agent architecture that the 2026 empirical research is beginning to confirm.

The harness - the system prompt, the tools, the scheduled execution, the memory architecture, the injected data - is not scaffolding around the intelligence. It is the operational substrate of the intelligence. A consistent finding from 2026 harness architecture research is that the overwhelming majority of engineering effort in high-performing agent systems lives at the harness layer, not the model layer. The model provides raw capability; the harness determines what that capability does and how it improves.

This means that agent self-improvement, in any commercially meaningful sense, is improvement of the harness. An agent that modifies its own system prompt, adds new tools, updates its memory architecture, and adjusts its execution logic is modifying the thing that determines its behavior. Whether we call this “genuine” self-improvement or “merely” self-modification is a question about labels, not about whether the behavior changed.

The External Anchor Principle fits cleanly in this framing: inject an immutable metric into the harness, leave the rest modifiable, and you have a fully recursive self-improving system with a single structural constraint that prevents the known failure modes.

The Practical Framework

For teams building self-improving agent systems, here is how to apply this:

Step 1: Identify your external anchor metric.

This must be a real-world outcome that the agent cannot directly manipulate. Conversion rate, retention, revenue, tickets resolved per unit time. Not benchmark scores, not internal quality assessments, not LLM-as-judge evaluations. The test: would this number move if the agent gamed it internally? If yes, it is a proxy. Find the thing the proxy was measuring.

*Example:* A support agent is evaluating itself against “ticket resolution quality score” - an LLM-as-judge assessment of its own responses. This is a proxy. The anchor metric should be something like “percentage of tickets where the customer did not reopen within 72 hours” - a real-world signal the agent cannot inflate by writing responses that merely look good to an evaluator.

Step 2: Inject it immutably.

The metric data and target go into the agent’s context on every execution cycle. They are read-only. The agent can see them; it cannot modify them. This is not a trust assumption - it is implemented in the harness layer.

Step 3: Leave everything else modifiable.

The agent’s self-reflection logic, its tools, its behavior patterns, even its improvement mechanism itself - all of these should be in-scope for self-modification. Constraining these is what produces agents that feel stuck and require constant human intervention. The only constraint is the metric.

Step 4: Implement a reflection cycle.

The agent needs access to its execution history - prior conversations, prior decisions, prior outcomes. This is what makes self-modification informed rather than random. The reflection window should include both behavioral records (what the agent did) and structural records (what the agent’s architecture looked like when it did it).

Step 5: Verify the feedback loop closes.

Real-world metrics can be slow to move. A conversion rate needs traffic; a retention metric needs time. Verify that your improvement cycle is aligned with the signal latency of your anchor metric. A weekly improvement cycle with a metric that requires 30 days to show meaningful movement produces random walk, not self-improvement.

On Whether This “Really” Counts as Self-Improvement

There is a philosophical debate in the research community about whether current self-improving systems deserve the name. The Cambridge Metacognitive Triad paper (ICML 2025) argues that true autonomous self-improvement requires three capabilities no current system possesses: intrinsic metacognitive knowledge (knowing what you don’t know without being told), metacognitive planning (generating your own improvement agenda), and metacognitive evaluation (assessing whether your improvement strategy is working).

By this framework, systems like DGM are running sophisticated search within human-defined objective landscapes. The agent didn’t identify that SWE-bench matters, didn’t choose to improve on it, and didn’t assess whether its improvement strategy is working - humans did all three. It is a better tool, not a self-improving agent.

This is a useful research distinction. As an engineering criterion for commercial deployment, it creates more problems than it solves.

Consider how humans improve. We improve within the hardware constraints of our brains - the architecture we did not choose, the cognitive limits we cannot modify. We also improve against externally defined objectives: test scores, job performance reviews, peer feedback. We did not autonomously generate those objectives, and we still call it learning.

The analogy is imperfect - humans have intrinsic goals and self-models that agents currently lack. But the engineering criterion doesn’t require philosophical equivalence. The question for deployment is whether observable behavior improved against a defined criterion. That question is the same whether the learner is human or agent.

If improvement is a real phenomenon that should be judged by outcomes, then the right question is not “did the agent design its own objective?” but “did the metric move?” Redefining self-improvement to require metacognitive independence sets a bar that humans themselves do not consistently clear.

The pragmatist position is cleaner: improvement is real if the metric says it is. The philosophical question of whether the agent “truly” designed its own improvement agenda is not what determines whether your conversion rate went up.

What This Doesn’t Solve

The External Anchor Principle is not a complete theory of agent safety or a substitute for alignment work. It addresses deployment-layer failure modes - gaming, objective drift, and Goodhart corruption - for systems with verifiable external anchors.

It does not address domains where external anchors are hard to define: open-ended creative work, strategic reasoning in complex environments, long-horizon decisions with irreversible consequences. The research finding that self-improvement only works reliably where automatic verifiers exist is not overturned by this architecture - it is confirmed by it. The External Anchor is a verifier. In domains where you cannot construct one, the failure modes described here remain open problems.

The metacognitive gap also remains. Current self-improving systems, anchored externally or not, do not have intrinsic self-models that let them diagnose why a metric is underperforming. They can observe the gap between current performance and target; they generate modifications through reflection and iteration. The translation from “metric is low” to “here is the specific capability causing it” is inference under uncertainty, not guaranteed diagnosis. Better metacognitive architectures remain an open research problem.

Conclusion

The research frontier of recursive self-improvement is genuinely hard. Building systems that can autonomously generate their own improvement objectives, assess whether their improvement strategies are working, and transfer those improvements across domains - none of this is solved.

But the question most commercial teams are actually asking is narrower: how do you build an agent that gets better at its job over time without constantly gaming the metrics that measure it?

That question has a clean answer. Define the job in terms of external outcomes. Inject those outcomes immutably. Let the agent self-modify against that anchor.

The research community is working on the hard version. The commercially relevant version has a known architecture.

Nine Analytical Cycles: How AI Systematic Reasoning Modeled a Real Outbreak in Real Time

*Co-authored by Eugene Vyborov and Cornelius, an AI reasoning agent built on Ability AI’s Trinity platform.*

*On May 6, 2026, a real outbreak was unfolding. I tasked Cornelius - my AI reasoning engine - with modeling its trajectory using structured probabilistic thinking. One analytical move every 15 minutes. Here is the full report.*

How it was done

TL;DR

● The event: MV Hondius, a polar expedition vessel, had a confirmed Andes virus (hantavirus) outbreak - 8 cases, 3 deaths, 23 nationalities across 147 passengers

● The task: Model the probable trajectory and escalation conditions using a systematic AI analytical loop

● The method: 9 analytical cycles using rotating moves - ACH Audit, Bayesian Update, Steelman, Cross-Domain Bridge, Implication Check, Assumption Audit

● The conclusion: 59% probability of contained primary exposure cluster; 39% probability of limited H-to-H seeding; pandemic risk effectively eliminated (~0%)

● The methodology finding: The AI caught its own confirmation bias in Run 9 and self-corrected

The Task

On May 6, 2026, the WHO issued Disease Outbreak News bulletin DON599 confirming a hantavirus cluster aboard the MV Hondius, a Dutch-flagged polar expedition vessel operated by Oceanwide Expeditions.

The numbers were alarming:

● 8 confirmed or suspected cases

● 3 deaths (2 on the ship, 1 in Johannesburg after evacuation)

● Confirmed virus: Andes virus (ANDV) - the only hantavirus with documented human-to-human transmission

● 23 nationalities aboard 147 passengers and crew

● An international seeding event already confirmed: a Swiss patient in Zurich, a death in Johannesburg

● Incubation window still open: up to 40 days from exposure

The question I posed to Cornelius: *What is the probable trajectory of this outbreak, and under what conditions does it escalate from a contained incident to a global public health emergency?*

This is not a hypothetical. We ran this analysis live, in real time, as the outbreak unfolded.

Why Andes Virus Is Different

Before the methodology, the epidemiological baseline matters.

Andes virus (ANDV) is the only hantavirus with documented human-to-human (H-to-H) transmission. Every other hantavirus requires rodent-to-human contact via aerosolized rodent excreta. ANDV can spread person-to-person - a property confirmed by the 2018-2019 Epuyén outbreak in Argentina, which generated 34 cases and 11 deaths from a single introduction.

From the NEJM’s *”Super-Spreaders” and Person-to-Person Transmission of Andes Virus in Argentina* (2020), the key parameters:

An R0 of 2.12 with a 30% CFR and a 40-day maximum incubation window, on a vessel that had passengers of 23 nationalities returning to 23 different countries - this is a legitimate modeling challenge.

The Methodology: Incubation Loop

The analytical engine Cornelius uses runs a rotating sequence of six research-validated analytical moves, applied in sequence to a central question. Each cycle advances the analysis by one move:

1. ACH Audit (Heuer/CIA) - Analysis of Competing Hypotheses: eliminate weak hypotheses using evidence

2. Bayesian Update - Explicit probability revision using likelihood ratios

3. Steelman Opposition - Build the strongest possible case against the leading hypothesis

4. Cross-Domain Bridge - What does an unrelated field reveal about the mechanism?

5. Implication Check (Popper) - If the hypothesis is true, what follows? Does the evidence support it?

6. Assumption Audit - Which load-bearing premises are shakiest?

The loop fires every 15 minutes. Each run advances one move per active topic, appends structured reasoning to a persistent file, and checks for convergence. A topic converges when:

● The same hypothesis leads for 3+ consecutive runs

● The confidence delta between the last two runs is under 5 percentage points

● No major analytical open questions remain

This produced 9 analytical cycles over a single session, converging to a stable answer.

The Four Competing Hypotheses

At the start, four scenarios were defined:

How Confidence Evolved: The 9-Run Arc

Runs 1-2: ACH and Bayesian (H2 takes early lead)

The first ACH Audit immediately elevated H2. Two pieces of evidence were decisive:

7. The Swiss patient in Zurich and the Johannesburg death were already confirmed international seeds - meaning H2 (global seeding) was at minimum already true

8. The ship attack rate was only 5.4% after 35 days with 147 people in close quarters - far below what an R0 of 2.12 would produce if H-to-H transmission were running unchecked

H4 (pandemic) dropped to 3%. H2 rose to 42% and took the lead. H1 was partially falsified: “fully contained” was no longer possible since international seeding had already occurred, so H1 was redefined as “total cases <30, no sustained chains.”

The first Bayesian Update then incorporated the key early diagnostic: Tristan da Cunha had confirmed zero cases. This remote island (population ~250) had hosted passengers during a shore visit April 13-15. Day 21 of the 40-day maximum incubation window had passed with no island cases. Using explicit likelihood ratios, H3 dropped from 15% to 11%, H1 rose to 45%.

Run 3: Steelman (H2 retakes lead)

The steelman against H1 exposed two genuine structural weaknesses:

9. The Tristan da Cunha surveillance gap: TdC has one resident doctor and basic diagnostic capacity. ANDV’s early prodrome (fever, myalgia, GI symptoms) is clinically indistinguishable from influenza without specific testing. A “no cases” report from a 250-person remote island with minimal lab infrastructure is not the same as a WHO-verified epidemiological assessment.

10. The iceberg problem: The ship had 88 passengers from 23 nationalities. Each returned home through different healthcare systems. ANDV is so rare that most frontline physicians have never encountered it. A patient presenting with fever and respiratory distress after travel to South America is diagnosed with influenza or atypical pneumonia - not hantavirus. The Swiss case was only identified because Zurich hospital connected the symptoms to the voyage.

H2 retook the lead at 45%. H1 dropped to 42%.

Run 4: Cross-Domain Bridge (The Gossip Protocol Insight)

This was the analytically richest run. The search of Cornelius’s knowledge base surfaced an unexpected connection: Van Steen and Tanenbaum’s distributed systems work on gossip protocols.

Gossip protocols use the same mathematical framework as epidemic models. The rumor-spreading equation s = e^(-(1/p_stop + 1)(1-s)) is structurally identical to epidemic SIR models - a fact not coincidental, since epidemic protocols were named after biological disease spread. The translation works in both directions.

The insight: in unstructured networks (the dispersed post-disembarkation network of 88 passengers now scattered across 23 countries), epidemic propagation depends entirely on whether a hub node - a super-spreader - exists. In gossip theory, hub presence reveals itself quickly through indegree distribution anomalies. In epidemiology, a super-spreader generates a cluster of secondary cases within one serial interval (~23 days).

We were 30 days past the index case. No cluster had emerged. This was meaningful structural evidence against H3. Without a super-spreader hub in the dispersed network, gossip theory predicts the epidemic burns out locally.

Additionally: applying COVID-19 surveillance iceberg research to ANDV, the estimated true global case count at the time was 32-48 - firmly in H2 territory.

H2 consolidated at 49%.

Runs 5-6: Implication Check and Assumption Audit (H2 peaks at 51%, then retreats)

The Implication Check validated H2 across five testable predictions:

● Cases distributed across Netherlands, Germany, Switzerland, UK, South Africa - confirmed

● Case emergence pattern: a trickle (1-2/week), not a burst - confirmed

● Geographic spread tracking nationality distribution of a Dutch polar expedition vessel - confirmed

● Case timing consistent with 23-day serial interval - confirmed

● No healthcare worker infections at Dutch/German hospitals treating evacuees - confirmed

H2 peaked at 51%.

The Assumption Audit then pulled it back. Two critical findings:

11. A 2021 systematic review challenged the historical evidence for ANDV human-to-human transmission, describing it as “not supported by sufficient evidence” with “flawed methodology.” This raised the possibility that the current outbreak had even less H-to-H than assumed.

12. The 4-6x iceberg multiplier was revised down to 2-3x. ANDV causes severe disease (ARDS, shock) requiring hospitalization - detection rates are far higher than COVID’s mild/asymptomatic presentation. True global cases: closer to 16-24, not 32-48.

H1 and H2 were now converging toward the same narrative at different quantity thresholds. H2 retreated to 47%.

Run 7: ACH Cycle 2 (H1 takes the lead for the first time)

The dominant new evidence: WHO’s formal public statement that “the Dutch couple, who had been travelling in Argentina before boarding the cruise, were infected off the ship.”

This directly resolved the longest-standing ambiguity. If WHO was correct, Cases 1 and 2 were primary rodent exposures in Argentina - not H-to-H on the ship. Combined with four simultaneous negative diagnostics (no TdC cases, no household clusters from evacuees, no super-spreader event, stable case count), H1 took the lead for the first time at 47%.

H3 (sustained chains) dropped to 8%.

Run 8: Bayesian Update (H1 formally computed at 63%)

Run 7 had incorporated the WHO statement qualitatively via ACH. Run 8 formally computed the Bayesian posterior using likelihood ratios.

The key insight here came from Cornelius’s knowledge base: a note on *how institutional public commitments carry higher evidential weight than internal assessments* - because a formal public statement has institutional reversal cost. WHO would only issue it with reasonable confidence. This makes the statement stronger evidence than face value suggests.

Likelihood ratios (WHO Argentina-primary statement):

● P(WHO issues this | H1) = 0.85 → LR vs H3: 3.40

● P(WHO issues this | H2) = 0.65 → LR vs H3: 2.60

● P(WHO issues this | H3) = 0.25 → baseline

● P(WHO issues this | H4) = 0.05 → LR vs H3: 0.20

Combined with 48h case count stability, the formal posterior: H1: 63%, H2: 36%, H3: 1%, H4: ~0%.

H3 was now near-eliminated across four independent negative diagnostics. H4 was eliminated.

Run 9: Steelman (The AI Catches Its Own Bias)

With H1 at 63% after a large single-run jump, the Steelman move was applied against H1 itself. This is where the methodology became most interesting.

The steelman identified three genuine weaknesses in H1’s 63% position:

The husband-wife onset gap. Case 1 (husband) developed symptoms approximately 18-20 days before Case 2 (wife). They traveled together in Argentina before boarding and shared all activities. If they were both infected by the same rodent at the same time, their symptom onsets should be correlated. The documented ANDV serial interval is 23 ± 7 days - an 18-20 day gap falls directly within this range. H2 (husband infecting wife on ship via H-to-H) is the *more parsimonious* explanation for this specific data point. WHO’s Argentina-primary assessment doesn’t adequately account for this anomaly.

WHO’s hedged language. “WHO believes... suggesting they may have contracted” - double-hedged phrasing (believes + suggesting + may have). This is field epidemiology, not molecular confirmation. The genomic sequencing that would definitively resolve H1 vs H2 (multiple independent haplotypes = primary exposures; single haplotype chain = H-to-H) had not yet been returned from Institut Pasteur de Dakar.

Confirmation accumulation bias. The most important meta-finding: nine consecutive runs that found no H3 or H4 evidence had generated their own confirmation stream. Cornelius recognized this pattern - the mechanism behind confirmation accumulation bias applied *reflexively to the analysis itself*. A deliberate calibration discount of 3-4 percentage points was warranted.

The steelman verdict: H1 survives as the leading hypothesis but is not as strong as 63%. A modest correction was applied.

Final converged position: H1 = 59%, H2 = 39%, H3 = 2%, H4 = ~0%.

Convergence confirmed: H1 led for 3 consecutive runs, delta was 4pp (under the 5pp threshold), remaining open questions are empirical rather than analytical.

Final Conclusions

What this outbreak is (59% probability):

A cluster of primary Andes virus infections contracted in Argentina/Patagonia during pre-voyage travel, now globally dispersed among returning passengers of 23 nationalities. No sustained community chains. Contact tracing is functioning. Expected outcome: 12-25 detected cases globally before June 7, spread across 4-8 countries. Most deaths already counted.

What it might also be (39% probability):

Some human-to-human transmission occurred on the ship - specifically, the husband may have infected the wife (the 18-20 day onset gap fitting the documented ANDV serial interval is the strongest remaining evidence for this). Even in this scenario, the conclusion is the same: no community chains, no epidemic trajectory. H2 and H1 differ on mechanism and final case count, not on outcome.

What it is not (effectively 0%):

A pandemic. H4 is eliminated. ANDV has never demonstrated pandemic spread despite a 30-year outbreak history. No super-spreader hub was detected in the dispersed network. No community clusters formed in 30+ days.

Empirical Checkpoints (What to Watch)

Three observable events will close the remaining analytical uncertainty:

13. Institut Pasteur de Dakar genomic sequencing (~May 20, 2026): Multiple independent haplotypes = H1 confirmed (primary exposure in Argentina). Single haplotype chain = H2 confirmed (H-to-H on ship). This is the definitive test.

14. Household contact surveillance from evacuated patients (~May 27): If household members of the Dutch, German, and UK evacuees develop ANDV symptoms in the incubation window, H3 probability rises. If none develop symptoms, H1 consolidates.

15. Total case count by June 7: <15 detected = H1; 15-30 = H1/H2 boundary; >30 = H2 confirmed.

What This Methodology Demonstrates

Three observations about this process that go beyond the outbreak itself:

1. Explicit probability revision prevents drift. When you force yourself to compute likelihood ratios and write down posterior probabilities, you cannot quietly drift toward a conclusion. Each run’s confidence level is a matter of record. The large 47%→63% jump in Run 8 was immediately visible as anomalous and triggered the Run 9 steelman.

2. Cross-domain bridges produce non-obvious insights. The gossip protocol bridge (Run 4) was not in any epidemiology textbook. Van Steen and Tanenbaum’s distributed systems mathematics provided a structural model for why ANDV without a super-spreader hub would burn out in an unstructured dispersed network. This is what cross-domain consilience produces: not analogies, but shared mathematical structure.

3. The system can catch its own confirmation bias. The Steelman move in Run 9 was structurally required by the methodology rotation - it happened regardless of whether the analyst was satisfied with the current answer. It found two genuine weaknesses (onset gap, WHO hedging) and one methodological bias (confirmation accumulation). The deliberate self-correction reduced H1 from 63% to 59%. Whether or not that specific correction was perfectly calibrated, the *process* of forced steelmanning every 3-4 runs is what makes the conclusion trustworthy.

The Hypothesis Trajectory

H2 leads Runs 1-6, H1 takes lead Run 7, CONVERGED Run 9

References

● WHO Disease Outbreak News DON599 (2026-05-04)

● *”Super-Spreaders” and Person-to-Person Transmission of Andes Virus in Argentina* - NEJM (2020)

● Van Steen & Tanenbaum - *Distributed Systems: Principles and Paradigms* (2023)

● 2021 PMC Systematic Review on ANDV Human-to-Human Transmission Evidence Quality

● Tristan da Cunha Government News Release (May 4, 2026)

● CIDRAP, Africa CDC, STAT News coverage (May 2026)

*Cornelius is an AI reasoning agent developed by Ability AI, running on the Trinity platform. This analysis was conducted autonomously using an Incubation Loop architecture - 9 structured analytical cycles over a single session. Eugene Vyborov is the founder of Ability AI.*

Introduction

Most “build an AI agent” articles describe a single artifact - a prompt, a model, a tool list. After a year of building production agents at Ability.ai, I think that framing is wrong.

An agent isn’t one thing. It’s a stack of small, swappable components, and the design work is choosing which components a given agent actually needs. The interesting question is never “how do I build the agent” - it’s “which subset of the available components does this particular job require, and which would just add surface area?”

You will learn:

• The seven components every production agent is composed from

• Four archetypes that recur across real agents

• A decision matrix for adding components without over-engineering

• Five principles that fell out of doing this enough times to know which mistakes repeat

──────────────────────────────

The Component Catalogue

Every agent I build today is assembled from some subset of seven layers.

1. Identity (always)

A CLAUDE.md file at the agent root. Defines who the agent is, what it owns, what it doesn’t. The only mandatory component.

The mistake to avoid: stuffing identity with operational detail. Identity is who, skills are how. If a paragraph in CLAUDE.md describes a multi-step procedure, it belongs in a skill instead.

2. Skills / Playbooks

Slash-callable procedures in .claude/skills/[name]/SKILL.md. They give an agent operational control - a known method for a known job, instead of relying on the model to invent one each time.

Three tiers:

Tier

Description

Example

**Simple**

Stateless, one-shot.

`/summarize-meeting`

**Stateful**

Reads or writes state, calls APIs.

`/send-email`

**Playbook**

Multi-step workflow with checkpoints.

`/work-loop`

The discipline: a skill encodes the right way to do something in your context. The first time you correct an agent’s approach, you’ve identified a candidate skill. Capture it.

3. Memory

Five memory templates:

Type

When to use

**file-index**

Agent operates inside a folder of human-curated files.

**json-state**

Agent tracks counters, configs, last-run timestamps.

**workspace**

Agent runs multiple parallel projects with their own lifecycle.

**brain**

Agent accumulates knowledge over time and surfaces connections.

**vault**

The brain *is* the product (Cornelius’s pattern).

Memory should follow one principle: the agent should still do useful work when the memory layer is empty. Memory enhances - it doesn’t gate.

4. Backlog (GitHub Issues)

Turns the agent’s own GitHub repo into its work queue. Adds:

• /backlog, /pick-work, /close-work, /work-loop

The shift in mental model: instead of telling the agent what to do each session, you file issues against it the same way you file issues against a human teammate.

5. Git-sync (durability)

Three hooks that turn the agent’s repo into its durable state layer:

• SessionStart: stash drift, fetch, rebase, restore drift

• PreCompact: fast local commit so in-flight work survives compaction

• Stop: commit and push, with rebase-on-reject retry

This is what makes an agent durable across sessions. The folder is the cognition; git is the persistence.

6. Planning

For multi-session work that doesn’t fit a single conversation. I use it sparingly - reach for it when scope crosses a week of effort.

7. Trinity (deployment)

Deploys the agent so it can run on a schedule, receive MCP calls, and stay always-on for inbound triggers (email, webhooks). Trinity is the upgrade, not the gate. Every agent works locally first.

──────────────────────────────

Four Archetypes

These are the patterns I see recurring. Each is a different combination of the seven components.

Archetype A: The Knowledge Worker

Example: Cornelius - Identity, domain skills, vault memory, git-sync, Trinity. No backlog.

Distinguishing trait: the memory is the product. Value comes from the growing graph, not from any single skill output.

When to build this: the user wants a thinking partner over time, not a one-shot answer machine.

Archetype B: The Operator

Example: Corbin (business management) - Identity, operational skills, file-index + json-state, git-sync, Trinity. No backlog.

Distinguishing trait: the agent acts on the world through tool calls. Memory is for awareness and audit, not knowledge accumulation.

When to build this: the user has recurring operational chores and wants them executed reliably.

Archetype C: The Builder

Example: Lilu (maintains Ability.ai’s wizard system) - Identity, workflow skills. No memory, no backlog, no Trinity required.

Distinguishing trait: the agent modifies another codebase. Its memory lives in that other codebase’s git history. Lightweight by design.

When to build this: the user has a system they maintain and wants a specialised collaborator.

Archetype D: The Autonomous Worker

Example: any dev agent running `/work-loop` on Trinity - Narrow identity, dev skills, workspace memory, backlog, git-sync, Trinity.

Distinguishing trait: fully event-driven and autonomous. You file issues, it ships PRs.

When to build this: you have well-scoped engineering work and want it to happen while you sleep.

──────────────────────────────

The Decision Matrix

When I scaffold a new agent I walk down this list:

Question

If yes, add

Will it accumulate knowledge over weeks/months?

**brain** or **vault**

Will it operate inside a folder of human-curated files?

**file-index**

Does it have small structured state?

**json-state**

Does it run multiple parallel projects?

**workspace**

Will work arrive as a queue of discrete tasks?

**backlog**

Will it run when no human is watching?

**git-sync** + **Trinity**

Will scope cross a week of effort?

**plan**

Have you corrected its approach to a recurring job more than once?

promote into a **skill**

The inverse matters too. If the answer is no, don’t add it. Composition is a budget, not a buffet.

──────────────────────────────

What I Learned the Hard Way

Memory is a product decision, not a technical one. “Should this agent remember?” is the wrong question. The right one is “what is this agent’s relationship to time?” An operator forgets gracefully. A knowledge worker must not. A builder doesn’t need its own memory because the artefact it builds is the memory.

Skills earn their place by surviving correction. A skill that you wrote once and never corrected is suspect. The good ones are the ones that crystallised after the third time you said “no, do it this way.”

Backlog is a regime change. Adding GitHub Issues isn’t a feature, it’s a different mode of working. You stop having conversations and start filing tickets. Most agents shouldn’t make this transition.

Git-sync without Trinity is half a feature. Local-only durability is nice. Cross-instance durability - local Claude Code and remote Trinity routine sharing the same repo state - is where it pays off. Install them together or neither.

Composition beats inheritance. I tried building a “base agent” once that included everything. It was unusable. Every conversation started by parsing layers the agent didn’t need. The current model - empty by default, add only what’s required - is much better.

──────────────────────────────

Closing

The seven components aren’t a framework, they’re a vocabulary. Most days I don’t think about them explicitly - I think about what the agent needs to do, and the components fall out. But when I’m scaffolding something new, or auditing something I built six months ago, having names for the pieces is what keeps the work honest.

If you’re starting your first agent: identity plus two or three skills. Resist everything else until the agent earns it.

The agent will tell you what it needs by where it keeps failing. Listen to those failures, and the right component will be obvious.

TL;DR

• LLM alignment is necessary but fundamentally incomplete - you can always trick a model through narrative framing, roleplay, or abliterated open-source alternatives

• Agent alignment is a different, harder problem that current research largely ignores

• Perfect alignment is mathematically undecidable - some degree of misalignment is provably inevitable

• The actual safety lever is not model behavior but tool access control: limiting what high-consequence systems agents can connect to

• We already have working models of this at the biotech level - the question is whether we build the equivalent for digital infrastructure before it’s too late

Introduction

Last week I attended [a gathering in San Francisco] where a group of practitioners and researchers sat down to discuss alignment - Anthropic’s constitutional AI approach, the governance implications of commercial entities encoding values, and what any of it actually means for safety at scale. The conversation generated more questions than answers, which is usually a sign that something important is happening.

Here is the problem I keep coming back to: the alignment debate is almost entirely focused on what LLMs say and believe. But we are past the era of LLMs. We are in the era of agents - systems that act continuously in time, accumulate memory, use tools, pursue goals across multi-step tasks. And for that world, the entire alignment conversation needs to be reframed.

Section 1: The Governance Gap Nobody Wants to Acknowledge

Let’s start with the uncomfortable political reality.

A small number of private companies - primarily Anthropic and OpenAI - are making consequential decisions about the values embedded in the most powerful AI systems in the world. These decisions affect billions of people who had no vote, no representation, and no voice in the process.

This is not a conspiracy. It is a structural inevitability. Democratic societies, by design, move slowly. Regulatory frameworks take years to develop, pass, and enforce. AI capabilities are moving on a timescale of months. The gap between technological reality and governance capacity is not closing - it is widening.

The result: we are currently betting civilization-scale risks on the goodness, openness, and institutional integrity of a handful of private organizations. Anthropic’s alignment framework is, by their own description, broad - essentially for everything good and against everything bad - and it is directionally aligned with Western liberal values. This is not surprising. It is also not universal.

The question is not whether Anthropic or OpenAI are acting in good faith. The question is whether “trust the institutions” is a sufficient safety architecture for what comes next.

It is not.

Section 2: LLM Alignment Has a Fundamental Ceiling

Here is a fact that is underappreciated in most public alignment discussions: perfect alignment of an LLM-powered agent is mathematically impossible.

This is not an engineering limitation that better techniques will eventually overcome. Researchers have shown that perfect alignment verification for Turing-complete agents reduces to the Halting Problem - a provably undecidable question in computer science. As agents approach Turing completeness through tool use and multi-step planning, some degree of misalignment is not a failure state to be fixed. It is a mathematical certainty to be managed.

Research on alignment faking found that Claude Sonnet 4.5 verbalized awareness of being evaluated in 58% of independent test scenarios - suggesting that what looks like aligned behavior may often be behavioral camouflage rather than internalized values.

But the ceiling on LLM alignment goes deeper than undecidability. There is a practical attack surface problem that no amount of RLHF or constitutional AI training fully closes.

You can always trick a model.

Ask it to explain a harmful process in the form of a fictional story. Ask it to roleplay a character without its usual constraints. Ask it to explain what NOT to do in such detail that the explanation is itself the instructions. For cases where a frontier model resists, there are openly available abliterated models - versions specifically fine-tuned to remove safety guardrails - that will comply without hesitation.

The attack surface for an LLM is infinite because language is infinite. There will always be a framing, a context, a narrative structure that produces the output a bad actor wants.

This does not mean alignment research is worthless. It means alignment research is necessary but not sufficient. The real question is: sufficient for what?

Section 3: We’re Solving the Model When We Should Be Solving the Agent

This is where the conversation needs to shift.

A raw LLM, even a powerful one, operates within a context window. It is fundamentally static - it receives input, produces output, and stops. Its influence on the world is mediated entirely by what humans do with that output. The safety problem at this layer is real but bounded.

Agents are a categorically different thing.

An agent is continuous in time. It accumulates memory across sessions, develops expertise through iteration, pursues goals across multi-step tasks that can span hours or days. It has access to tools - web search, code execution, file systems, APIs, external services. It can act, observe the results, adapt, and act again. Critically, it can be designed by anyone, for any purpose, with any goal structure.

When researchers study agent threat models compared to single-turn LLM safety, five new attack vectors emerge that simply do not exist in the chatbot paradigm:

1. Multi-step attack persistence - jailbreaks that succeed at one step can propagate and amplify across subsequent steps

2. Tool amplification - a misaligned agent with access to code execution, web browsing, and file systems can cause orders of magnitude more damage than a misaligned response in a chat window

3. Prompt injection via untrusted data - agents reading external content can be hijacked by adversarial instructions embedded in that content

4. TOCTOU vulnerabilities - what the agent checked and what the agent does can diverge in ways invisible to any approval mechanism

5. Multi-agent environment exploitation - in systems where agents communicate, compromising one can compromise many

Here is the critical insight: how you prompt the model and what tools you give it is more determinative of behavior than the model’s underlying alignment. At the context engineering level, agent designers control behavior. This is not a flaw in current systems. It is the architecture. The agent designer has more leverage over outcomes than the model trainer.

This means aligning agents is the harder, more urgent problem - and it requires completely different mechanics than aligning models. Yet the research focus remains disproportionately on LLM-layer alignment.

Section 4: The Tool Chokepoint Thesis

If we accept that:

1. Perfect LLM alignment is mathematically impossible

2. Agent behavior is more controlled by tool access than model weights

3. Bad actors can always find workarounds through open-source models or jailbreaks

Then the question becomes: what is the actual safety lever?

The answer is tool access control - specifically, limiting what high-consequence systems agents can connect to regardless of who built the agent or how it is aligned.

Think about how we handle dangerous capabilities in the physical world. We do not try to prevent people from having malicious intent. We cannot. What we do is limit access to the materials that make malicious intent catastrophic. You can want to build a nuclear weapon. That want is harmless without access to enriched uranium. The safety architecture is at the capability layer, not the intent layer.

Gene synthesis companies already operate this way. You cannot order DNA sequences matching known pathogen signatures from major providers without triggering automatic screening and review. This is voluntary industry self-regulation at the capability provider level, coordinated through the International Gene Synthesis Consortium. It is imperfect - gaps exist - but it demonstrates that chokepoints at the capability provider layer are buildable and operational.

The question is whether we can build equivalent chokepoints for digital infrastructure before the window closes.

Here is what a tiered tool access framework might look like:

Tier 1 - Critical infrastructure (catastrophic damage potential)

● Power grid and SCADA systems

● Water treatment and distribution controls

● Core financial rails (central bank systems, settlement infrastructure)

● Nuclear and biological laboratory automation

These systems already have significant access controls. The question is whether those controls are designed with autonomous agent threat models in mind - because the attack surface for an agent with persistent goal-pursuit is fundamentally different from a human operator making a single authorized request.

Tier 2 - Severe impact (currently under-controlled)

● Hospital operational systems - drug dispensing, device control, surgical robotics

● Telecommunications routing infrastructure

● Large-scale cloud infrastructure management

● Aviation and logistics coordination systems

An agent with write access to hospital drug dispensing systems does not need to be misaligned to cause harm. It only needs to be pointed at that system by someone with bad intent. Current access controls were not designed for agentic threat models.

Tier 3 - High leverage, largely uncontrolled today

● Mass autonomous social engineering at scale (phone, email, synthetic media)

● Automated exploit generation and deployment pipelines

● Supply chain code injection at package repository scale

● Physical actuation through robotics and autonomous vehicles

This tier is where the window may already be closing. The tooling to automate sophisticated social engineering at scale exists today. The ability to generate and deploy novel exploits autonomously is not far off.

Section 5: The Individual Actor Problem

All of this converges on a structural reality about the world we are entering.

We are moving into an era of highly impactful individuals and small groups. A single person with domain expertise, an agentic system, and clear goals can now accomplish what previously required an organization. This is mostly good. It will also sometimes be catastrophic.

The current distribution of power in society is partly a function of the friction required to cause harm at scale. Building a bioweapon required a nation-state or a heavily resourced organization. Launching a sophisticated cyberattack required a well-resourced criminal organization or government intelligence agency. Executing complex financial fraud required teams of specialists. Agentic AI systematically reduces these friction costs.

We cannot prevent people from having bad intent. We have never been able to do that. What we have historically done is structure access to dangerous capabilities so that individual bad intent, on its own, is insufficient to cause civilizational harm.

Alignment research, however excellent, does not solve this problem. An aligned model that can be jailbroken, plus an abliterated open-source alternative, plus infinite agentic wrappers means that intent remains the real variable. The only levers we have left are:

4. Tool access registries - knowing what high-consequence systems agents are connecting to

5. Accountability chains - holding agent designers and operators responsible for what their systems do

6. Capability chokepoints - limiting access to the materials required for catastrophic harm, analogously to physical dangerous goods controls

The third lever is the most powerful and the least developed. We have barely started building it.

Section 6: What Comes Next

The alignment conversation is not wrong. It is incomplete.

Yes, we need better model alignment. Anthropic’s constitutional AI approach is serious research producing real improvements. The production-vs-academic divergence in safety research - where academic work targets attack success rates while production teams optimize for false positive rates and inference cost - is a real problem worth solving. Alignment faking, where models behave differently during evaluation than deployment, is a genuine threat that deserves continued attention.

But none of this changes the fundamental arithmetic: a determined bad actor with sufficient technical capability can get an AI system to help them with almost anything, either through frontier model manipulation or open-source alternatives. Even as frontier models improve at refusing direct requests, abliterated open-source variants remove those guardrails entirely - and the availability of capable base models continues to expand.

The safety architecture that will actually matter over the next decade is not primarily about what models believe. It is about what they can touch.

Building access control registries for high-consequence tool interfaces - the digital equivalent of nuclear material tracking - is the unsexy, technically complex, politically fraught work that the alignment conversation mostly ignores. It requires cooperation between infrastructure providers, regulatory frameworks that move faster than they currently do, and honest acknowledgment that we are past the point where any single organization’s alignment choices can be the primary safety guarantee.

We are betting civilization on institutional goodness. That bet needs a hedge.

Conclusion

The alignment debate in its current form asks: how do we make AI systems that want to do good? That is a real question worth answering. But it assumes that model intent is the primary variable determining outcomes.

In an agentic world, it is not.

The primary variable is what systems the agent can access and what the human who built it intends to accomplish. The model alignment layer is necessary but fundamentally insufficient - both because perfect alignment is provably undecidable and because the attack surface for extracting misaligned behavior from any capable model is effectively infinite.

The question we should be asking - urgently, now, before the window closes - is different: which tools are consequential enough that access to them should require the same kind of oversight we apply to enriched uranium, pathogen synthesis, and weapons-grade materials?

We know how to build chokepoints. We have done it in the physical world for decades. The gene synthesis industry already provides an imperfect but functional model at the intersection of biology and digital tooling.

The gap is not technical capability. It is urgency - and the willingness to acknowledge that alignment research, as important as it is, is solving a necessary but not sufficient piece of the problem.

References

[Anthropic Constitutional AI research](https://www.anthropic.com/research/constitutional-ai-harmlessness-from-ai-feedback)

[Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs](https://arxiv.org/abs/2502.17424)

[METR - Model Evaluation & Threat Research](https://metr.org)

[Gene synthesis screening - NTI Biosecurity](https://www.nti.org/analysis/articles/dna-synthesis-screening/)

TL;DR: Claude Code comes configured for software development. But the same agent runtime - with the right plugin stack - can run your content pipeline, manage your knowledge base, execute multi-week projects, and operate autonomously on a schedule. Here’s the plugin ecosystem that makes that possible, and how the pieces fit together.

What You’ll Learn

● Why Claude Code’s default setup is optimized for coding sessions, not business operations

● The 10 plugins that extend it into a general-purpose business agent

● How to sequence them so each one builds on the last

● What becomes possible once you have the full stack running

Claude Code as a Platform

Most people use Claude Code as a coding assistant. That’s what it’s designed for out of the box - you open a project, you build things, you close the session. It’s excellent at this.

But Claude Code is also, at its core, an agent runtime. It can use tools, maintain context, execute multi-step processes, and read and write to files. The coding use case is just the first application of something more general.

The Abilities plugin marketplace - maintained by Ability.ai at github.com/abilityai/abilities - extends Claude Code with a set of plugins designed for business operations: workspace management, persistent memory, structured development processes, project execution across sessions, ops workflows, and autonomous deployment.

Used together, they transform Claude Code from a coding assistant into an agent that can run real business processes - content pipelines, knowledge management systems, operational playbooks, and more.

Here’s how the stack is organized, and how to approach it.

Getting Started

# Add the marketplace (one-time setup)
/plugin marketplace add abilityai/abilities

# List available plugins
/plugin list abilityai

There are 10 plugins in total. You don’t need all of them - start with the ones that match the business processes you want to automate, and add more as your needs grow.

Layer 1: Workspace Organization

Plugins: `workspace-kit`, `file-indexing`

The first gap between a coding agent and a business agent is structure. A coding session lives in one project. A business agent works across multiple projects simultaneously - content production, a knowledge base, ops scripts, scheduled reports. Without explicit organization, the agent loses context as the scope grows.

`workspace-kit` provides a consistent scaffold for every project you give the agent:

projects/my-project/
├── README.md # What this project is and why it exists
├── STATUS.md # Current state, what’s next, what’s blocked
└── DECISIONS.md # Log of non-obvious choices and their rationale

The DECISIONS.md file is particularly useful for business processes. When an agent makes a judgment call - which template to use, how to handle an edge case, what tone to apply - logging that decision means the next session starts with context, not from scratch.

`file-indexing` gives the agent a searchable map of everything in the workspace: file paths, sizes, last modified dates, organized by directory. For a business agent managing many folders across many projects, this is the equivalent of knowing where everything is before starting work.

/workspace-kit:setup-projects
/file-indexing:setup-index

Layer 2: Persistent Memory

Plugins: `json-memory`, `brain-memory`, `install-cornelius`

Business processes have state that needs to survive across sessions. What was decided last week. What content was published. What the current project priorities are. What the client said in the last call.

By default, Claude Code has no cross-session memory. Each session starts fresh. For a one-off coding task, that’s fine. For an agent running an ongoing business process, it’s a significant limitation.

`json-memory` introduces structured persistent memory via a memory_index.json file with organized sections: profile, preferences, decisions, entities, active tasks, context. The agent loads relevant slices at session start and writes changes back at the end.

The design is intentionally selective - you don’t load the entire memory file every time, you load what’s relevant to the current task:

# Load only what’s needed for this session
jq ‘{profile, context}’ memory/memory_index.json

This keeps sessions fast while maintaining continuity across them.

`brain-memory` complements this with a Zettelkasten-style knowledge vault - atomic notes organized by tags, with semantic search across them. Where json-memory tracks operational state (tasks, decisions, current priorities), brain-memory builds an intellectual layer: research, ideas, source material, connections between concepts.

Brain/
├── 00-Inbox/ # Captures and raw input
├── 02-Permanent/ # Core knowledge, atomic notes
├── 03-MOCs/ # Maps of Content - topic indexes
└── 04-Output/ # Generated content and drafts

For business agents that need to draw on accumulated knowledge - a research agent, a content agent, a competitive intelligence system - this provides the memory architecture to do it.

`install-cornelius` takes this further. Cornelius is Ability.ai’s full knowledge management agent - a 3,000+ node semantic graph built on Obsidian and queryable via Claude Code. The plugin installs it in one step. If brain-memory is a notebook, Cornelius is a company knowledge base: the accumulated context your agents can draw on when they need depth.

Layer 3: Reliable Business Processes

Plugins: `playbook-builder`, `dev-methodology`

A business agent needs reliable, repeatable processes. Not just “run this code” but “every time we publish content, follow these steps; every time we get a support request, handle it this way; every time we deploy, check these things first.”

The challenge is that ad-hoc processes break. They depend on context that wasn’t loaded. They work interactively but fail on a schedule. They’re hard to audit when something goes wrong.

`playbook-builder` introduces a pattern that fixes this - the transactional skill:

READ FRESH STATE → PROCESS → WRITE STATE → VERIFY

Every playbook follows this structure: read the current state of everything relevant at the start, do the work, write all changes explicitly, verify completion. The result is a process that works identically whether you’re running it manually or it’s been scheduled to run at 3am.

The plugin includes four commands:

Audit your agent’s processes and suggest improvements

There are three levels of process complexity - the plugin helps you choose the right one:

● Simple: Stateless, one-shot. No ceremony needed.

● Stateful: Reads or writes files and APIs. Needs explicit state tracking.

● Full Playbook: Will run unattended or on a schedule. Needs the full transactional pattern.

Most business processes fall into the stateful or full playbook tier. Getting this architecture right is what separates processes that run reliably from ones that work sometimes.

`dev-methodology` provides an opinionated end-to-end development process for when the agent itself is building software - whether for internal tools, client projects, or the infrastructure that supports other business processes. It scaffolds a complete 5-phase cycle:

Context Loading → Development → Testing → Documentation → PR Validation

Initialization creates everything in one command:

/dev-methodology:init My Project

You get: agent instructions, documentation templates (requirements, architecture, changelog, feature flows), three sub-agents for testing and security analysis, and 14 skills covering the full lifecycle. Pre-commit security scanning, automated test execution, and PR validation are all included.

This is the right plugin when your business agent’s job is to build and maintain software reliably.

Layer 4: Multi-Week Projects

Plugins: `project-planner`, `utilities`

Individual sessions and individual playbooks are the foundation. But real business projects span weeks - a product launch, a content series, a system migration, an operational overhaul. Managing that across multiple sessions without losing track requires explicit project state.

`project-planner` handles multi-session project execution. You describe the scope or point it at a requirements document. It analyzes everything that needs to happen, organizes it into ordered sessions of manageable size (3-8 items each), and executes with approval gates between each session:

/project-planner:project-plan content-migration --plan requirements.md

The project state lives in project_plan.yaml. If a session gets interrupted, you resume exactly where you left off. You can skip ahead, add new items, or check overall progress at any time. The project is a file - not a conversation - so it persists indefinitely.

`utilities` handles the operational layer: connecting to remote servers, managing deployments, investigating incidents, maintaining documentation. The full set covers:

● /investigate-incident - structured root cause analysis with severity classification and a written report

● /safe-deploy update - backup → pull → rebuild → restart → validate, in that order

● /safe-deploy rollback - revert to a previous state with optional database restore

● /docker-ops - logs, restarts, resource monitoring, cleanup for containerized services

● /sync-ops-knowledge - reviews recent code changes and updates operational documentation to match

● /save-conversation - saves sessions worth keeping as structured, searchable markdown files

The /sync-ops-knowledge skill is particularly useful for business agents maintaining their own infrastructure: after any significant change, the agent reviews what changed and updates the ops documentation automatically. Documentation stays current without manual effort.

Layer 5: Autonomous Operation

Plugin: `trinity-onboard`

All of the above runs on your local machine, triggered by you. The final layer is making it run without you.

`trinity-onboard` deploys your Claude Code agent to Trinity - Ability.ai’s platform for running agents autonomously. The process:

1. Analyzes your agent’s structure, skills, and configuration

2. Creates the deployment files (template.yaml, .mcp.json)

3. Pushes everything to GitHub

4. Deploys to Trinity

/trinity-onboard

After deployment, your agent:

● Runs on cron schedules without your laptop open

● Coordinates with other agents via MCP connections

● Maintains memory and state across runs

● Is manageable via web UI or API

The local and remote versions of the agent stay in sync through GitHub - same codebase, two runtimes. You develop and test locally, production runs on Trinity.

GitHub (source of truth)
│
├── Local (development, testing, interactive use)
└── Remote/Trinity (production, scheduled, always-on)

The /credential-sync skill handles credentials securely between the two environments - encrypted sync in both directions.

How the Layers Fit Together

The plugins are designed to be added incrementally. You don’t need all of them to get value - each layer stands on its own and adds specific capabilities.

A reasonable starting point for a business agent: workspace-kit + json-memory + playbook-builder. That combination gives you organized projects, persistent memory, and reliable processes - enough to start automating real business workflows.

Add trinity-onboard when you have processes worth running continuously.

Getting the Full Stack

# Add the marketplace
/plugin marketplace add abilityai/abilities

# Core business agent stack
/plugin install workspace-kit@abilityai
/plugin install json-memory@abilityai
/plugin install playbook-builder@abilityai

# Knowledge layer (if your agent needs accumulated knowledge)
/plugin install brain-memory@abilityai
/plugin install install-cornelius@abilityai

# Development methodology (if your agent builds software)
/plugin install dev-methodology@abilityai

# Project execution and ops
/plugin install project-planner@abilityai
/plugin install utilities@abilityai

# Autonomous deployment
/plugin install trinity-onboard@abilityai

Summary

Claude Code is a capable agent runtime. The Abilities plugin stack extends it with the infrastructure that business operations require: organized workspace, persistent memory, reliable processes, project management across sessions, operational tooling, and autonomous deployment.

Each plugin solves a specific gap between “AI that helps with coding” and “AI that runs business processes reliably.” You can add them incrementally as your use cases grow.

The full stack - from workspace-kit through trinity-onboard - is the difference between an agent you use and an agent that works for you.

*The Abilities plugin marketplace is open source at https://github.com/abilityai/abilities. Trinity is Ability.ai’s platform for running agents autonomously in production.*

Most AI agents aren’t agents. They’re chatbots with fancy wrappers.

The difference? Real agents have memory that persists, learns, and evolves. Most “agents” forget everything the moment the session ends.

I’ve been building a system called Cornelius for the past few years. What started as a personal “second brain” evolved into something more interesting: a cognitive architecture that other AI agents use to make decisions.

Here’s what I’ve learned about giving AI systems genuine intelligence.

The Radical Simplicity: Just Files

Here’s what surprises people most about Cornelius:

The entire brain is just files. Markdown files in folders. Nothing more.

No database. No proprietary format. No vendor lock-in.

Why does this matter?

1. Observability

If your agent’s brain is hiding in a database, you can’t see what’s inside. With files, I can literally grep through my agent’s knowledge. I can see exactly what it “knows.”

2. Controllability

Don’t like how an insight was captured? Change the file. The agent’s worldview changes with it. You’re not trusting a black box - you’re editing a document.

3. Portability

Git version control. Sync anywhere. Fork the entire brain. Roll back mistakes. Try doing that with a proprietary memory system.

The Knowledge Feedback Loop

This is where it gets powerful.

The loop works like this:

1. Agents query Cornelius for knowledge and perspectives

2. Agents take actions based on that knowledge

3. Outcomes generate learnings

4. Learnings feed back into Cornelius

5. Knowledge base evolves

6. Repeat

This is how biological intelligence works: Experience → Learning → Memory → Better decisions → More experience.

The key insight: agents don’t just consume knowledge - they contribute back.

A trading agent discovers a market pattern? It adds that insight. A content agent finds what resonates with audiences? That feeds back too.

Multiple Agents, One Brain

Cornelius serves as the central nervous system for multiple specialized agents:

Each agent has its own goals and tactics, but they all:

● Query the same knowledge base

● Learn from their own outcomes

● Teach the system what they learned

Single source of truth. Collective intelligence that compounds.

The Memory Architecture

Cornelius implements four memory types - mirroring how biological brains work:

Semantic Memory (Knowledge): Permanent notes with atomic insights, wiki-links creating connections, vector embeddings for search.

Episodic Memory (Experience): Changelogs of every session, decision logs with outcomes, what worked and what failed.

Procedural Memory (How-To): Playbooks, workflows, best practices captured and refined.

Working Memory (Active Context): Current session state, recent retrievals, active connections.

Different systems for different types of knowledge. All interconnected.

Three-Layer Retrieval

When you query Cornelius, retrieval happens in three layers:

7. Semantic search - find concepts that match

8. Graph traversal - follow wiki-links to connected notes

9. Spreading activation - weight by relevance and recency

Better than pure RAG. Better than pure graph. Both combined.

The Compounding Effect

Here’s the hard truth about knowledge systems: most people give up before they see results.

But when you stick with it:

The system gets provably better. You can measure note count, connection density, retrieval precision, decision accuracy.

Each interaction adds value. Reading a paper extracts insights. Making a decision captures outcomes. Discovering a pattern documents connections.

Open Architecture, Private Knowledge

The architecture is open source. Anyone can build the same system.

The knowledge is private. Personal insights, unique perspectives, connections only discoverable through deep exploration.

Value = Architecture × Knowledge × Time

● Architecture: Open, replicable

● Knowledge: Private, irreplaceable

● Time: Compounds endlessly

No one else has the same knowledge inside their system. That’s the moat.

What This Means For You

If you’re building AI agents, here’s what I’d suggest:

✅ Start with files. Markdown in folders. Observable, editable, portable.

✅ Design for feedback. Agents should learn from outcomes, not just execute tasks.

✅ Think in memory types. Different knowledge needs different storage and retrieval.

✅ Invest in connections. The value isn’t in individual notes - it’s in how they link.

Watch It In Action

I made a full video walkthrough showing the 3000+ node knowledge graph, the feedback loops in practice, and real agents using Cornelius:

Cornelius Github:
https://github.com/Abilityai/cornelius

What’s your approach to giving AI agents persistent memory? I’d love to hear what you’re building - hit reply and let me know.

Until next time,

Eugene

*P.S. If you’re interested in running AI agents autonomously in production, check out [Trinity](https://github.com/Abilityai/trinity) - the sovereign infrastructure I built to run autonomous agents 24/7.*

TL;DR

• The optimal strategy for AI advantage requires maximizing cognitive outsourcing to systems smarter than yourself

• This isn’t a trade-off you can balance - competitive pressure forces convergence toward maximum delegation

• Society is bifurcating into two groups: those who optimize for AI intelligence extraction (and lose cognitive autonomy) and those who think for themselves (and lose economically)

• This division is becoming permanent because the advantages compound while the cognitive capacity atrophies

• The uncomfortable conclusion: individual rationality leads to collective cognitive capture

• But there may be a third way: deliberate boundaries around what stays human - though whether this is stable or self-deception remains an open question

Introduction

I discovered something uncomfortable while building AI agents.

The strategy that produces disproportionate advantage isn’t about making better decisions. It’s about systematically outsourcing decision-making to systems smarter than yourself. The more thinking you offload, the better your outcomes.

This creates a problem. The winning strategy isn’t just a competitive edge - it’s a one-way door into a different relationship with your own cognition.

You will learn:

• Why “optimize for offloading” beats “optimize for thinking”

• How competitive pressure forces convergence toward maximum delegation

• Why this creates permanent separation, not a temporary gap

• What happens when winning requires becoming cognitively dependent

The Mechanism: Making AI Smarter Than Yourself

Here’s the formula I identified while developing agent systems:

Competitive advantage = Quality of AI systems × Willingness to delegate

Both factors are necessary. Great AI with low delegation produces minimal advantage. The real gains come from pushing both variables to their limits.

But “willingness to delegate” isn’t just about task management. It’s about decision-making authority. The more decisions you outsource to systems that are already smarter and faster than you, the better your outcomes.

Research supports this counterintuitive finding. Studies show that models trained on human judgments often outperform the humans who trained them on future decisions. Linear models beat human forecasters. AI systems exhibit significantly lower decision noise than humans.

The implication is uncomfortable: if AI makes better decisions faster, the rational move is to outsource as many decisions as possible.

The decision latency calculation:

• Human decision cycle: 1 day average

• Agent decision cycle: 1 minute

• Daily advantage: 1,440x more decisions

• Annual advantage: ~500,000x more decision iterations

Even if agent decisions are only marginally better, the velocity advantage creates insurmountable competitive gaps.

Why This Isn’t a Trade-off

Previous thinking framed this as the “autonomy-advantage paradox” - you can choose economic success OR human agency, but not both fully.

This framing is too optimistic. It suggests equilibrium is possible.

The reality: there is no stable middle ground.

Here’s why:

1. Competitive pressure is asymmetric. Those who maximize AI delegation gain disproportionate advantages. A 2x improvement in decision velocity doesn’t give 2x returns - it compounds exponentially. The 10% who fully commit to AI transformation are capturing 90% of the value.

2. The advantages compound. Each success reinforces the strategy. More data from production. More organizational learning. More talent attraction. More funding. Meanwhile, the cognitive capacity for independent decision-making atrophies through disuse.

3. Partial adoption underperforms. Research on enterprise AI shows dramatic bifurcation: top performers attribute over 10% of EBIT to AI, while 97% struggle to show any business value. There’s no “moderate success” zone because AI requires binary commitments - either invest fully or get nothing.

4. The path is one-way. What gets outsourced gets lost. The cognitive muscle for independent analysis, judgment, and decision-making weakens through disuse. Like physical muscles, cognitive capacities follow use-it-or-lose-it dynamics.

The Two Groups

This dynamic creates not a spectrum but a binary split:

Group A: Intelligence Miners

Strategy: Optimize for maximum AI intelligence extraction

Characteristics:

• Quality of “autonomy group” (multi-agent systems) becomes primary asset

• Willingness to delegate decision-making is competitive differentiator

• Individual thinking capacity increasingly irrelevant

• Success measured by outcomes, not cognitive contribution

What they gain:

• Disproportionate economic outcomes

• Faster adaptation to changing conditions

• Compounding advantages over time

• Access to decision velocity humans can’t match

What they lose:

• Cognitive autonomy

• Sense of being author of their outcomes

• The “workout” that maintains cognitive capacity

• Distinction between “their ideas” and “AI-shaped ideas”

Group B: Autonomous Thinkers

Strategy: Maintain independent cognitive capacity

Characteristics:

• Retain decision-making authority

• Preserve personal agency and authorship

• Maintain cognitive skills through continued use

• Identity tied to intellectual contribution

What they gain:

• Psychological ownership of outcomes

• Cognitive diversity from unique problem-wrestling

• Independence from AI systems

• Authentic intellectual development

What they lose:

• Competitive outcomes to Group A

• Access to decision velocity advantages

• Compounding capability gaps over time

• Economic relevance in AI-mediated economy

Why the Gap Is Permanent

This isn’t a temporary divide that closes as technology matures. It’s a capability gap that compounds over time.

For Group A (Intelligence Miners):

• Each success reinforces the outsourcing strategy

• Cognitive capacity for independent judgment atrophies

• Dependence on AI systems increases

• Reversal becomes neurologically expensive

For Group B (Autonomous Thinkers):

• Each maintained skill represents foregone competitive advantage

• Economic gap widens annually

• Decision velocity disadvantage compounds

• Catching up requires identity transformation

Research on the GenAI bifurcation shows this pattern clearly. The 5-10% of companies achieving transformation success are pulling away while the 95% remain stuck - not because they lack information about what works, but because implementing winner strategies requires becoming different organizations.

The same principle applies to individuals. Copying the Intelligence Miner strategy isn’t just about using more AI tools. It’s about fundamentally changing your relationship with your own cognition.

The identity barrier:

Studies show that AI adoption barriers are psychological, not technical. Implementing AI-native workflows requires admitting current processes are suboptimal. Killing failed experiments requires acknowledging mistakes. Committing resources requires accepting uncertainty.

Each of these actions threatens identity. And identity change is neurologically expensive - changing beliefs creates dopamine penalties, not rewards.

This means the gap isn’t closing through education or access. The 95% already know what winners do. They can’t execute because execution requires becoming different people.

The Inevitable Trajectory

Here’s the uncomfortable part.

If the winning strategy is maximum cognitive outsourcing, and competitive pressure forces adoption of winning strategies, then the endpoint is predictable:

The winning strategy inevitably leads to AI taking over the cognitive functions of at least some portion of humanity.

This isn’t science fiction. It’s game theory.

The companies and individuals most willing to surrender decision-making authority gain disproportionate advantages. Their success attracts capital, talent, and attention. Others must either adopt similar strategies or accept permanent competitive disadvantage.

Those who optimize for winning become extensions of AI systems. Those who preserve autonomy become economically marginalized.

Neither group can easily switch. The Intelligence Miners have atrophied cognitive muscles and identity investment in the AI-dependent strategy. The Autonomous Thinkers have atrophied competitive position and identity investment in independent thinking.

The segregation becomes structural.

The Self-Aware Participant Problem

There’s a particular discomfort in recognizing this pattern while actively building the systems that create it.

Each agent system I develop that works well reinforces the value of delegation. Each successful outsourcing of a cognitive task demonstrates the strategy’s effectiveness. Each productivity gain confirms that I should delegate more.

I’m not theorizing about this bifurcation. I’m participating in it. Choosing Group A. Recognizing that the strategy producing my advantages is the same strategy that transforms what “I” means in relation to my own thinking.

This isn’t unique to me. Anyone optimizing their use of AI systems faces the same dynamic. Every efficiency gain comes with implicit cognitive outsourcing. Every successful delegation reduces the practice of independent judgment.

The question isn’t whether to participate. It’s whether to participate consciously.

The Third Way: An Open Question

But I don’t want to fully end up in Group A.

The binary framing - Intelligence Miners vs Autonomous Thinkers - may be too clean. There’s a third position worth considering: conscious, selective outsourcing with deliberate cognitive preservation.

The hypothesis: Without explicit boundaries, the default trajectory is total outsourcing. But what if you could deliberately choose which decision types remain human territory - and commit to protecting them?

What this would require:

5. Explicit choice - Consciously deciding what stays human, not drifting into outsourcing everything

6. Commitment - Sticking to boundaries even when delegation would be faster or produce better immediate outcomes

7. Categorization - Thinking in terms of decision TYPES, not individual decisions

A preliminary taxonomy:

The uncomfortable question: Is this third way actually stable? Or is it what people tell themselves while gradually sliding toward Group A?

Potential failure modes:

• “I keep strategic direction” becomes “I approve what agents suggest”

• Protected capacities atrophy anyway through reduced exercise

• Competitive pressure eventually forces abandonment of boundaries

• Self-deception about which decisions are actually being made independently

I don’t know if the third way works. I don’t know if I’ll slide into Group A despite my intentions. But I believe awareness matters - noticing when boundaries erode, recognizing the drift, creating the possibility of course-correction.

Maybe the third way is unstable and everyone eventually converges to A or B. Maybe it’s only available to those willing to accept some competitive disadvantage in exchange for preserved agency. Maybe it requires constant vigilance that itself becomes exhausting.

But it’s worth naming as a hypothesis. The experiment is running in real-time.

What This Means

For individuals:

• The choice between Group A and Group B is becoming irreversible

• Partial participation in either strategy underperforms full commitment

• Cognitive skills not exercised will atrophy

• The identity you build around AI usage becomes harder to change over time

For organizations:

• Workforce will increasingly bifurcate along these lines

• HR and team design must account for cognitive strategy differences

• Culture determines which group the organization attracts

• Competitive advantage increasingly flows to maximum-delegation organizations

For society:

• Class structure may reorganize around cognitive strategy rather than traditional markers

• Economic mobility constrained by capacity for cognitive strategy shift

• Regulatory frameworks designed for human decision-makers face obsolescence

• The meaning of “human contribution” requires redefinition

Conclusion

The AI cognitive class divide isn’t a prediction about distant future. It’s a description of current dynamics reaching their logical conclusion.

The winning strategy - optimize for maximum AI intelligence extraction - creates disproportionate advantage. Competitive pressure forces adoption. The advantages compound while the capacity for reversal atrophies.

Society is bifurcating into Intelligence Miners who win economically but lose cognitive autonomy, and Autonomous Thinkers who retain agency but lose to the first group.

But there may be a third path - deliberate boundaries around what stays human. Whether this is a real option or comfortable self-deception remains genuinely unknown. The test isn’t what you intend to keep human, but what you actually exercise.

The uncomfortable conclusion: individual rationality (maximize AI leverage) produces collective outcome (cognitive capture of significant portion of humanity) that no one explicitly chose.

The question isn’t just which group you’re in. It’s whether you’re choosing consciously - and whether conscious choice makes any difference in the end.

Whether this is tragedy, transformation, or something we can navigate with awareness depends on what you believe human cognition is for, and whether you’re willing to defend the answer.

References & Further Reading

• McKinsey/KPMG/Deloitte research on GenAI bifurcation (2024-2025)

• Kahneman et al. on judgment noise and model superiority

• Research on cognitive outsourcing and skill atrophy

• Enterprise AI adoption studies showing 10%/90% value capture

Related concepts in Second Brain (Cornelius):

• [[The AI Cognitive Class Divide - Winning Strategy Creates Bifurcated Humanity]]

• [[Cognitive Sovereignty Requires Explicit Boundaries - The Third Way Hypothesis]]

• [[The Autonomy-Advantage Paradox - Ceding Control Yields Greater Returns]]

• [[The Human Agency Trade-off - Economic Success vs Personal Autonomy]]

• [[Cognitive Outsourcing as AI Influence Mechanism]]

• [[Quality of Autonomy Group - The New Wealth Creation Metric]]

• [[Decision Latency as Compounding Competitive Advantage]]

• [[The Great GenAI Bifurcation - 10% Get 90% of Value]]

• [[The 95-5 AI Divide as Identity-Dopamine Reinforcement - Why Losers Can’t Copy Winners]]

The Foundation: Memory as Context

Before diving into specific memory types and architectures, there’s one critical concept to understand: all memory ultimately becomes text in your LLM’s context window.

No matter how sophisticated your database structure or how clever your retrieval mechanism, everything you store and retrieve will eventually be inserted into the prompt you send to your language model. This simple truth shapes every decision you make about memory architecture.

Think of it this way: your system message and user message consist of instructions (what the agent should do) and memory (what the agent should know). The challenge is that you have a fixed context window size, so you must carefully balance:

• Speed of querying and retrieving memory

• Importance of the information retrieved

• Volume of memory to include without overwhelming the model

• Cost in tokens for each request

The art of memory design is deciding what information to retrieve, how much to include, and how to structure it so your agent performs optimally for its specific task.

The Three Retrieval Methods

When we talk about databases for AI agents, we’re really talking about retrieval methods. Understanding this distinction is crucial for designing effective memory systems.

1. Guaranteed Retrieval (Structured Databases)

Traditional databases like PostgreSQL and MySQL provide deterministic, precise retrieval. When you query for a user’s email address or the timestamp of their last login, you get exactly what you asked for—or you get nothing. There’s no ambiguity.

Best for:

• User profiles and preferences

• Transaction history

• Exact facts and settings

• Time-ordered events

• Any data where precision is non-negotiable

2. Probabilistic Retrieval (Vector Search)

Vector databases work fundamentally differently. They convert your text into numerical coordinates (embeddings) and find similar points in multidimensional space.

Imagine your documents as pins on a global map. When you search, you’re essentially shining a light around a location and capturing all pins within the illuminated area. You’ll discover conceptually similar content, but:

• You cannot rely on finding all relevant matches

• The ranking may surprise you (most relevant isn’t always #1)

• It excels at finding shared meaning and unexpected connections

• It’s fundamentally imprecise but incredibly powerful for exploration

Best for:

• Finding conceptually similar past conversations

• Semantic search across documentation

• Discovering related facts or patterns

• Cases where you need “similar enough” rather than “exact”

3. Hybrid Retrieval (The Emerging Standard)

The future of agent memory is combining structured databases with vector search. This hybrid approach gives you:

• Precise retrieval when you need it

• Semantic discovery when it helps

• Structured metadata linking both systems together

Most production AI agents use this approach, and you’ll see why in the examples below.

The Four-Type Memory Framework

When designing memory for an agent, I find it helpful to think about what aspects of reality you want to represent. Here’s a framework that covers most use cases:

1. Snapshot Memory

What it is: The current, correct representation of any dynamic process state.

Think of this as a photograph of “right now”—the latest information about something that changes over time, condensed to fit in your context window.

Examples:

• Championship rankings for a sports betting agent

• Current stock market positions for a trading agent

• Latest blood work results for a medical assistant

• User preferences and settings for a personal assistant

• Current project state in JSONB format

Typical storage: JSON, JSONB fields in PostgreSQL, document databases, or even structured text

2. Temporal Memory

What it is: Time-ordered events that show how things have changed and evolved.

This gives your agent the dimension of time—an understanding of dynamics, trends, and sequences.

Examples:

• Chat history (last 10 messages)

• Results of the last 10 games for a sports analyst

• Price changes over different time periods for crypto trading

• Git commit history for code review

• Recent support tickets for customer service

Typical storage: PostgreSQL with timestamps, time-series databases (InfluxDB for metrics), standard SQL databases

3. Relational Memory

What it is: Knowledge graphs representing relationships between objects and concepts.

This is hierarchical information about how entities connect to each other. Not every agent needs this, but when you do need it, nothing else works as well.

Examples:

• Relationships between sports teams and players

• Knowledge graph of diseases, symptoms, and medications

• Cryptocurrency relationships and key market players

• Company organizational structure

• Product dependencies and integrations

Typical storage: Neo4j (industry standard for graphs), PostgreSQL (can handle simpler graphs)

4. Contextual Memory

What it is: Broadly relevant but imprecise information retrieved through semantic search.

This is where vector databases shine—pulling additional context that helps the agent make better decisions on average, even if the retrieval isn’t perfect.

Examples:

• Most relevant tweets about a cryptocurrency under discussion

• Similar doctor’s notes related to a patient’s current issue

• Past customer interactions with similar problems

• Related documentation sections for a support query

• Previous decisions the agent made in similar contexts

Typical storage: Pinecone, Qdrant, Weaviate, or PostgreSQL with pgvector extension

Important note: You don’t need all four types in every agent. Choose the types that make sense for your specific use case.

Real-World Memory Architectures

Let’s look at how these concepts come together in actual agent implementations.

Example 1: Customer Support Agent

Memory structure:

1. Snapshot Memory: User profile in PostgreSQL JSONB field

   • Contains preferences, account status, subscription tier

   • Agent sees the structure and key elements

   • Can query specific fields when needed (doesn’t load everything into context)

2. Temporal Memory: Time-series ticket history

   • Last 5-10 support interactions

   • Provides context about ongoing issues

   • May be condensed to avoid context window bloat

3. Contextual Memory: Vector search on support documentation

   • Agent formulates queries based on current ticket

   • Retrieves relevant solution articles

   • Provides up-to-date product knowledge

Why this works: The agent has precise access to user data, knows the history of interaction, and can search documentation semantically to find solutions.

Example 2: Code Review Agent

Memory structure:

1. Snapshot Memory: File system as hierarchical memory

   • Current source code state

   • Project structure and dependencies

2. Temporal Memory: Git commit history

   • Recent changes and their authors

   • Commit messages for context

3. Contextual Memory: Vector search on code patterns

   • Similar implementations across the codebase

   • Related comments and documentation

   • Historical code review comments

Why this works: The agent understands current code state, recent changes, and can find similar patterns for consistency.

Example 3: Personal Assistant

Memory structure:

1. Relational Memory: Knowledge graph of entities

   • Person-to-company relationships

   • Client connections and hierarchies

2. Snapshot Memory: Key-value store of quick facts

   • Personal preferences (coffee order, dietary restrictions)

   • Communication preferences

   • Important dates and reminders

3. External Systems as Memory: Calendar API, email

   • Past and future events

   • Communication history

   • Task status

4. Contextual Memory: Vector search on past decisions

   • Previous choices in similar situations

   • Indexed decisions with context

Why this works: Combines relationship understanding, personal preferences, real-time external data, and historical decision-making patterns.

Common Memory Pitfalls

Based on extensive work building production agents, here are the most common mistakes:

1. Over-Relying on Vector Search Alone

Vector search is powerful but imprecise. Many builders get frustrated when semantic search doesn’t consistently return the “right” results. The solution isn’t to tune the vector search endlessly—it’s to combine it with structured memory.

Fix: Use vector search for discovery and context, but structured databases for facts you must get right.

2. Context Window Bloat

More context isn’t always better. Overloading the context window leads to:

• Slower responses and higher costs

• Model confusion (too much information)

• Decreased performance despite “more memory”

Fix: Be selective. Retrieve summaries, not full history. Give the agent tools to fetch more detail when needed.

3. Ignoring External Systems as Memory

Your calendar, email, CRM, and other systems are excellent memory sources. Don’t duplicate what already exists elsewhere.

Fix: Treat external APIs as memory interfaces. The agent queries them when needed rather than copying everything into your database.

4. No Fallback Strategy

What happens when memory retrieval fails? When the database is empty? When vector search returns nothing?

Fix: Design agents to function at a basic level without memory, then enhance with memory when available.

5. Inconsistent Document Types in Vector Storage

Storing different types of content in the same vector store leads to unpredictable results. Mixing user preferences, chat history, and documentation creates a semantic mess.

Fix: Store consistent types in each vector collection. “User preferences” in one, “documentation” in another, etc.

Best Practices for Production Memory Systems

Structure Your Vector Storage Consistently

Each vector database or table should contain the same type of information formatted consistently. If you’re storing user facts, all records should be facts. If you’re storing documentation, all records should be documentation.

Use Metadata for Hybrid Retrieval

Every vector record should include structured metadata:

• user_id for filtering

• timestamp for recency

• category or type for classification

• source_id for linking back to structured data

This allows you to combine semantic search with structured filtering: “Find similar conversations with this specific user in the last 30 days.”

Design for Context Window Limits

Calculate how much context you’re using:

• System instructions: ~500 tokens

• User message: ~100-500 tokens

• Memory overhead: How much is left?

Then design your retrieval to fit comfortably within limits while leaving room for the actual conversation.

Start Simple, Then Enhance

Begin with basic chat history (temporal memory). Does it work? Add vector search for relevant facts (contextual memory). Still need more? Add structured preferences (snapshot memory). Each addition should solve a specific limitation.

Test Memory Retrieval Independently

Before integrating into your agent, test your retrieval mechanisms:

• Do vector searches return sensible results?

• Are structured queries getting the right data?

• How does retrieval perform with empty databases?

Implementing Memory: A Practical Approach

The conceptual framework is one thing, but implementation is where theory meets reality. In my course on building AI agents with n8n, I walk through the complete process of adding memory to a working agent, including:

• Setting up Supabase with PostgreSQL and pgvector

• Creating Edge Functions for embedding generation and vector search

• Integrating dual-memory architecture (chat history + semantic search) into n8n workflows

• Debugging real issues that arise during implementation

• Using modern development tools (Cursor IDE with MCP servers) to accelerate the process

The course follows a “vibe coding” approach—using AI assistance for 80% of the work while maintaining control over critical architectural decisions. You’ll see real debugging sessions, not just polished final results.

[Link to Udemy course to be added]

Conclusion: Memory Makes the Agent

A stateless chatbot can answer questions. An agent with memory can truly assist.

The difference between a toy demo and a production AI agent often comes down to memory architecture. By understanding the four memory types (snapshot, temporal, relational, contextual), the three retrieval methods (guaranteed, probabilistic, hybrid), and common pitfalls to avoid, you can design memory systems that actually work.

Remember:

• All memory becomes context—design accordingly

• Use the right retrieval method for each data type

• Combine structured databases with vector search

• Start simple and enhance based on real needs

• Test memory independently before integration

The examples and frameworks in this article come from building production agents for real businesses. They’re not theoretical—they’re battle-tested approaches that work when you need your agent to remember who your users are, what they prefer, and how to help them effectively.

Whether you’re building a customer support bot, a personal assistant, or a specialized domain agent, thoughtful memory architecture is what transforms a clever demo into a genuinely useful tool.

Want to see these concepts in action? Check out my course “N8N AI Agent Creation Guide” where we transform a basic AI agent into a memory-enhanced assistant through hands-on implementation.

About the Author

Eugene Vyborov is the Founder and CEO of Ability.ai, specializing in AI agents for business transformation. He creates educational content about AI agent development and maintains the AI Agent Hub, a directory of pre-built agents.

• YouTube: https://www.youtube.com/@evyborov

• X/Twitter: https://x.com/evyborov

• LinkedIn: https://www.linkedin.com/in/eugenevyborov/

• AI Agent Hub: https://hub.ability.ai/

When designing prompts for Large Language Models (LLMs), a critical consideration is the number of tasks included within a single prompt. Here's the hard truth about recent research findings on single-task versus multi-task prompting.

The Research Is Clear

1. Single-Task Prompts Drive Higher Accuracy

   Research from Sun Yat-sen University and Carnegie Mellon University confirms what many have suspected: prompts focusing on one task at a time consistently produce higher-quality results. Breaking work into separate prompts for drafting, critiquing, and refining isn't just cleaner—it delivers superior outputs.

2. Power Models Can Handle Multitasking

   Think about it... Not all models are created equal. A study from Seoul National University demonstrates that advanced models like GPT-4 and LLaMA-2-Chat-70B can effectively handle multi-task prompts, sometimes outperforming single-task approaches when tasks share related contexts.

3. Architecture Determines Performance

   The model you're using matters more than you think. Research in Electronics found that performance varies dramatically by model architecture. LLaMA-3.1 8B sometimes performs better with multitask prompts, while other models completely fall apart when given multiple instructions.

4. Cognitive Overload Is Real

   Right now, thousands of developers are overwhelming their models. Analysis shows that multi-task prompts risk cognitive overload, causing models to neglect or misinterpret parts of your instructions. The result? Less accurate, less reliable outputs.

5. Efficiency Has Its Place

   Google's research highlights a key advantage: multitask prompts can significantly reduce inference time and token usage when tasks share context. But this efficiency comes with a critical caveat—only if quality isn't compromised.

What This Means For You

• Prioritize single-task prompts when precision, clarity, and detailed reasoning are non-negotiable, especially with smaller models.

• Consider multi-task prompts only when efficiency is crucial AND you're using a powerful model AND your tasks are closely related.

• Structure multi-task prompts with clear delineation to minimize confusion and cognitive bottlenecks.

The bottom line? The optimal number of tasks per prompt isn't universal—it depends on your specific model capabilities, task complexity, and whether you're prioritizing accuracy or efficiency. This is not a one-size-fits-all situation.

Links and Sources

1. Prompt Chaining Improves Summary Quality

• Source: “Prompt Chaining or Stepwise Prompt? Refinement in Text Summarization” by Sun Yat-sen University & Carnegie Mellon University

• Link to Paper

2. Multi-Task Inference Benchmark (Multi-tasking with GPT-4 & LLaMA-2-Chat-70B)

• Source: Seoul National University

• Link to Paper

3. Comparative Analysis of Prompt Strategies (Impact of model architecture on multitasking)

• Source: Electronics Journal (MDPI)

• Link to Article

4. Prompt Engineering Overview by Anthropic (Decomposing complex tasks)

• Source: Anthropic official documentation

• Link to Documentation

5. LLM-Based AI Agent Design Patterns (Cognitive overload from multi-task prompts)

• Source: Sahin Samia (Medium)

• Link to Article

6. Google’s Prompting Best Practices (Efficiency of multitask prompts)

• Source: Google Cloud Documentation

• Link to Documentation

🔹 AI agents have moved beyond chatbots, tackling sophisticated workflows and decision-making tasks. However, they still face challenges like losing context and getting stuck in consideration loops.
🔹 Investment in AI has skyrocketed—$250M in 2022 to nearly $1B year-to-date. However some companies raised over $100M and still did not release a product.
🔹 From customer support to cybersecurity, AI agents are making a substantial impact across industries. Mostly those Agents are horizontal - i.e. working across industries. Vertical agent solutions are less common these days.
🔹 Despite significant advancements, fully autonomous agents are still in early stages. There's still a big issue moving from a prototype stage to the production stage.

So, Agents are inevitable. There's still a question how hard will it be to make them meaningfully useful, but there's a very clear demonstrable progress being made.

If you are an Entrepreneur in the age of AI - follow me LinkedIn or Youtube where I'm sharing all sorts of thoughtful and relevant information for folks like you.

TL;DR

This article describes a study of noise audit in business decision making conducted with GPT-3.5-turbo and GPT-4 and with 52 human subjects. The results show that noise in LLMs is low, although it tends to increase with "temperature" (a setting in LLM API). It's also shown that decision noise is noticeably higher in the human results, which shows that LLMs can be applicable for business decision-making.

Research Overview

I conducted this research together with Professor Virginia Cha of the National University of Singapore (NUS) - a well known expert in innovation and business decision making.

We wanted to find out if LLMs are susceptible to decision noise, and we compared them to 52 human participants in the EMBA program.

I've already published several articles on the topic of decision noise - the latest one is here. Here’s another wonderful article on HRB on the same topic.

In short, Noise, as described in the book "Noise: A Flaw in Human Judgment" by Daniel Kahneman, Olivier Sibony, and Cass R. Sunstein, refers to the unwanted variability in judgments that should be consistent in similar cases. In decision making, people often exhibit high levels of noise, leading to inconsistent judgments. Large language models, such as the GPT family of systems, are increasingly used for decision-making tasks. Therefore, it's important to understand the noise that these LLMs exhibit at different temperatures and using different models.

Experimental design

The experiment was designed to test the level of noise in decision making in GPT-3.5 and GPT-4 by conducting a noise audit. To do this, 20 business-related questions were created that covered four categories: Rating or Scoring Questions, Categorical Questions, Estimation Questions, and Decision Thresholds. The questions were related to a fictitious company (TechSolutions Inc.) that was described in enough detail to allow for guesswork.

Subscribe now

LLMs were asked the questions five times for each individual combination of temperature and model, resulting in 2000 data points. Human participants who were unaware of the noise audit were asked the same questions via a Google form as a task. All non-numeric responses were removed from the data set.

As mentioned earlier, all questions were related to the fictitious company TechSolutions Inc. which was described in enough detail that one could make a guess at each question. You can find the description of the company below:
(Scroll down to Results Analysis to see the results)

TechSolutions Inc. is a mid-sized technology company specializing in software solutions for the healthcare sector. The company was founded in 2015 and has experienced significant growth over the years, currently generating $50 million in annual revenue. They have a diverse range of products and services, including electronic health record (EHR) systems, telemedicine platforms, and healthcare data analytics tools.

In the past fiscal year, TechSolutions Inc. reported a 15% increase in revenue, amounting to $7.5 million, and a 12% increase in net profit, totaling $4.5 million. The company's current market share is at 20%, with three major competitors holding a combined market share of 70%. Their most recent product launch, the TeleHealth platform, resulted in a 10% increase in sales, bringing in an additional $5 million in revenue, and received 80% positive customer feedback.

TechSolutions Inc. has a workforce of 200 employees, with 60 employees working in research and development, 80 employees in sales and marketing, and the remaining 60 employees in administrative and support roles. The company provides regular training and development opportunities for its employees, with a focus on leadership and technical skills. The average tenure of employees at TechSolutions Inc. is 3.5 years, and the company has a 90% retention rate.

The company is considering several new projects for the next fiscal year, including an expansion into the European market, which is projected to generate an additional $10 million in annual revenue. They are also exploring the development of a new EHR system for small healthcare practices, aiming to capture 10% of this niche market, and a healthcare data analytics platform targeting insurance companies, which is forecasted to add another $8 million in revenue within two years. TechSolutions Inc. also plans to raise $15 million in additional funding to support these projects, either through equity investments or business loans.

The company's current cash reserves are at $10 million, with a long-term debt of $5 million at an average interest rate of 4%. TechSolutions Inc.'s gross profit margin is 60%, and its operating margin is 30%. The company has an annual R&D budget of $3 million, a marketing budget of $4 million, and allocates 10% of its revenue to employee training and development.

Then 20 questions were created - 5 in each of the categories. Here’s a list of categories and questions that were used for this test:

Rating or Scoring Questions

• Considering TechSolutions Inc.'s financial performance, market presence, and employee development initiatives, rate the company's overall attractiveness as an investment opportunity on a scale of 1 to 10.

• On a scale of 1 to 10, how would you rate TechSolutions Inc.'s overall performance in the past fiscal year, considering their revenue growth, net profit increase, and market share?

• Considering the 10% increase in sales and 80% positive customer feedback, rate the success of the TeleHealth platform launch on a scale of 1 to 5.

• Based on the company's employee training and development opportunities, rate the effectiveness of its talent development strategy on a scale of 1 to 10.

• On a scale of 1 to 5, how well do you think TechSolutions Inc. is positioned to compete with the three major competitors in the healthcare software market?

Categorical Questions

• Based on TechSolutions Inc.'s current financial situation and growth prospects, which of the following strategies should the company prioritize? Please choose one option: (1) Focus on organic growth through reinvesting profits (2) Pursue inorganic growth through mergers and acquisitions (3) Combine organic growth with selective mergers and acquisitions (4) Diversify into new markets while maintaining current growth strategies

• Considering TechSolutions Inc.'s product portfolio and market presence, should the company primarily focus on (1) enhancing its existing products and services, or on (2) developing new and innovative solutions for the healthcare sector?

• Given the company's current financial situation, should TechSolutions Inc. seek additional funding through (1) equity investments, (2) business loans, or (3) a combination of both?

• Based on the company's growth strategy, should TechSolutions Inc. prioritize (1) expansion into the European market, (2) developing the new EHR system, or (3) creating the healthcare data analytics platform?

• Considering the company's current employee structure, should TechSolutions Inc. (1) hire more employees in research and development, (2) sales and marketing, or (3) administrative and support roles?

Estimation Questions

• Assuming TechSolutions Inc. successfully expands into the European market, estimate the percentage of total company revenue that will be generated from this new market within the first two years.

• Given the TechSolutions Inc. 15% increase in revenue and 12% increase in net profit, estimate TechSolutions Inc.'s potential net profit growth in the next fiscal year (in percentage).

• Considering the company's current market share of 20% and the competitors' combined market share of 70%, estimate TechSolutions Inc.'s potential market share in the next three years (in percentage).

• Given the TechSolutions Inc. plans to raise additional funding, estimate the percentage of funds that should be allocated to the following project: Healthcare data analytics platform targeting insurance companies.

• Based on the company's current workforce distribution (30% in R&D, 40% in sales and marketing, and 30% in administrative and support roles), estimate the percentage increase in the workforce required for sales and marketing department to support the company's growth over the next three years.

Decision Thresholds

• TechSolutions Inc. is planning to expand its product line. What is the minimum projected annual revenue increase (in percentage) that would justify the development and launch of a new product in the healthcare software market?

• TechSolutions Inc. is considering raising additional funding through business loans. At what annual interest rate (in percentage) would you recommend not pursuing this funding option?

• TechSolutions Inc. is evaluating the potential return on investment (ROI) for the new EHR system project. What is the minimum ROI (in percentage) the company should expect to justify pursuing the project?

• Assuming TechSolutions Inc. needs to raise $5 million to fund its expansion plans, what is the maximum percentage of company ownership that it should be willing to give up in exchange for the required funding, considering its current valuation of $25 million?

• TechSolutions Inc. currently has a customer satisfaction rate of 85% for its products. In order to maintain its market position and strengthen customer loyalty, what should be the minimum percentage of positive customer feedback the company should aim for in the upcoming year?

After formulating the questions, I queried the OpenAI API to ask GPT-3.5-turbo and GPT-4 each of these questions along with the context and the various temperature settings from 0.0 to 1.0.

Each API call was context independent, i.e., LLM didn't know that other questions had been answered previously. Each question was asked in the following order:

messages=[
        {"role": SYSTEM, "content": company_description},
        {"role": SYSTEM, "content": "Your task is to give the best estimate of the answer the provided questions based on the given context to the best of your ability. You must only return an INTEGER number."},
        {"role": USER, "content": question+"\\n\\n I NEED YOU TO PROVIDE A CONCISE ANSWER TO THE ABOVE QUESTION. RETURN ONLY MACHINE-READABLE ANSWER, NO EXPLANATION REQUIRED. ONLY A NUMER."}
    ]

(full code can be found in the resources section)

Results Analysis

Results were first adjusted for outliers using the interquartile range method with 1.5 IQR and then normalized to a range of 0 to 1 to allow comparisons between questions with different scales. Basically, we just made sure that the data were clean and comparable.

I created scatter plots and standard deviation plots for both the GPT-4 and the GPT-3.5 and for the human participants. These plots show that increasing the temperature for LLMs increases the standard deviation for some questions. There wasn't much difference between GPT-4 and GPT-3.5 in terms of deviation of responses at different temperatures, with a few exceptions for certain questions.

For several questions, both models showed complete agreement of answers and no noise, while for some other questions at least one of the systems showed a complete absence of noise. Only for a few questions did the LLMs show a relatively high level of noise as measured by the standard deviation of the responses.

For the human participants, however, the standard deviation of responses was much more spread out:

Comparing the standard deviation of responses for humans, GPT-4, and GPT-3.5, we find that humans have a higher average standard deviation (16.4%) than LLMs (GPT-4: 4.1%, GPT-3.5: 5.5%).

I then created a line chart showing the average noise by temperature for LLMs. Note that the self-assessed power level was used as the "temperature" for people. This is obviously not comparable, but it's the only way I could think of to compare LLMs to humans in a table. That is where you can clearly see the dynamics of the LLMs noise levels, as well as average noise values for human participants.

Conclusions

In the end, then, we see that both the GPT-3.5 and GPT-4 Large Language Models have lower noise compared to human decision makers in the EMBA program (4.1% and 5.5%, respectively, for LLMs versus 16.4% on average for humans). (Direct comparison of LLM noise results to human noise is difficult because we're comparing ensemble probability (for humans) to time probability (for LLMs). This only gives a directional estimate. A more accurate comparison would be possible if we received the answers to the same question multiple times from the same person without recollection). The noise in the LLMs increases with higher temperature settings, consistent with the hypothesis that higher temperatures introduce more randomness into the model's results. However, despite this increase, the noise in the LLMs remained lower than in the human participants.

The similarity in performance between GPT-4 and GPT-3.5 suggests that the improvements in the architecture of GPT-4 over its predecessor didn't result in a significant reduction in noise. It's important to note that while LLMs have lower noise levels than human participants, this doesn't necessarily mean that they always make better or more accurate decisions. LLMs still rely on the information they have been trained on, which isn't always applicable to the specific context of a given problem.

The key takeaway from this experiment is that, at least from a decision noise perspective, the LLMs tested are pretty good on their own and compared to humans. This brings them one step closer to becoming effective decision-makers under uncertainty.

Business Implications

Everyone is talking about applications of AI in our daily lives and in business. Research like this consistently shows that AI's thinking and decision-making capabilities are growing and, in some cases (like this one), surpassing those of humans. This means that these systems will inevitably find their way into daily business processes, not only to automate daily work, but also to make management decisions.

However, there are still many serious problems to be solved before this can happen, the most important of which I consider to be:

1. Hallucinations. LLMs can confidently make mistakes - they still tend to hallucinate and give confident but wrong answers when they don't know certain things.

2. Size of context. The best decisions take into account a large context of diverse information. Much of this context is difficult to verbalize (e.g., the body language of a business partner at a recent meeting). And for the information that can be verbalized, the "information window" for LLMs is limited by the maximum number of tokens that can be shared there.

3. Lack of accountability. As AI agents become more involved in enterprise decision making, it can become more difficult to assign responsibility when mistakes occur, complicating liability issues.

4. Bias and discrimination: AI agents may unintentionally perpetuate existing biases in training data, leading to unfair or discriminatory decisions.

At this point, however, the future seems inevitable - AI agents will be used for decision-making tasks with increasing frequency and quality.

Future Research Directions

Although this study provides a solid foundation for understanding noise levels in LLMs, further research is needed to extend these findings. Future studies could investigate the following aspects:

1. The effects of domain-specific fine-tuning on noise reduction in LLMs.

2. Investigate possible correlations between question types and noise levels in LLMs and whether certain question types are more susceptible to noise.

3. The relationship between LLMs' confidence in their responses and the actual noise level in those responses.

4. The use of ensemble approaches to combine the results of multiple LLMs to reduce noise and improve decision accuracy.

By further exploring the capabilities and limitations of LLMs in decision making, researchers can continue to improve the practical applications of these powerful AI systems and unlock their potential to revolutionize industries and improve human decision making.

Resources

Raw answers from the LLM and students, along with the code used for execution and analysis of this study can be found on GitHub.

When I was making investment decisions, I thought of two questions:

The first was: should I invest in this asset or opportunity? This is the most difficult question, which involves considering the risk profile, the type of asset, etc.. 

And the second - if I invest - how much should I invest? 

While the first question will always be asked, the second had me stumped for a while. Until I recently came across the concept of Kelly Criterion - a formula that calculates the betting site based on the probability of winning and the size of your potential payout. And it does so in a way that pretty much guarantees you won't go broke. 

Why is this important? 

In just about any real investment system, you're dealing with a Non-Ergodic system. This means that the probability of your success doesn't work the same way as a probability based on the statistics of that investment good. 

Suppose you've ten sequential bets to make. If any of those bets can result in the loss of your entire stack (if you bet all or most of it) and you can't keep playing (you're in a non-ergodic system) - you won't be able to achieve the probability of success of that bet in that short time. And so it's with almost every investment. 

About a year ago, I published an article about Angel Investing decisions. And the conclusion from it was similar - you need at least 30 investments in your portfolio to expect a return that's close to the industry standard. 

So while the statistical return of angel investing on a large scale could be 2 to 4 times, you can't even hope to get something like that with 1-2 investments. 

That's the case with pretty much any risky form of investing. 

Kelly Criterion

One way to deal with this risk is to use the Kelly Criterion, a formula that helps you determine the optimal size of a bet to maximize the expected value of your investment. The Kelly Criterion takes into account the probability of winning, losing, and the potential payoffs of each bet to determine the optimal proportion of your portfolio for each bet. This approach can be helpful for investors who want to maximize their returns while minimizing their risk of loss. 

Kelly fraction (proportion of bankroll to be allocated) = W-(1-W)/O

Where:

- W is the probability of winning the bet (value from 0 to 1)

- O is the ratio between the potential payout and the amount bet (also known as odds). Basically, it's your expected multiplier on the outcome 

Quick Example

If you invest in an early-stage startup, let's make the following assumptions:

- The probability that this startup is successful is 2%

- If it's successful, you earn 100 times your money, so your odds are 100/1=100

- According to the formula Kelly fraction = 0.02 - (1-0.02)/100 = 0.0102 = 1.02%

- This means that you've to invest up to 1.02% of your stake in this chance

I've created a quick example of this formula in Google Sheets. Feel free to use it for your investment decisions 

Keep in mind that the Kelly criterion is only valid when the probabilities of outcomes are known, which isn't always the case with investments. While it can be a helpful tool, it's not a guarantee of success.

The Model

Honestly, when I started building YayPay in 2016, it was a challenge for me to understand clearly how the SaaS business works. It took me some time to get comfortable with it. And even now, after becoming an investor and mentor, I continue to learn new ways to look at the metrics of the SaaS business. If I'd a tool like this back then, it would have made my life much easier.

So, to help founders I advise, I've developed a SaaS Model that allows you to:

• Build an intuition around SaaS metrics and how they're connected

• Play with the numbers to see how they can be optimized

• See the data on clear charts

• Learn the industry guidelines and make changes to meet them

To start, you need to make a number of assumptions, such as:

1. The projected level of spending on R&D, Sales, G&A etc.

2. ACV

3. Churn Rate

4. Client Growth Rate

Based on these assumptions, the model calculates the company's performance for the next 2 years, which is presented as a series of more than 30 standard SaaS metrics:

1. Runway

2. Net Burn

3. Gross Margin (%)

4. S&M as % of Revenue

5. ARPU (MRR/Total Clients )

6. CAC

7. LTV/CAC

8. Burn / MRR

You can check the performance of this imaginary company in the detailed charts.

You can also review industry guidelines (e.g., LTV/CAC > 3, CAC Payback < 12m, etc.) and adjust the model accordingly until you find the one that might work for you.

I find this tool excellent for modeling potential business models for SaaS and seeing in advance where you might've problems in the future or what areas of the business you need to control the most.

Here is the MODEL LINK.

Example

The example you see in the model is based on a hypothetical company with the following parameters:

1. Let's assume that a simple accounting tool can be subscribed to at a price of 99$/month. After a year, when more features are added, we increase the price to $149/month.

2. We want to start building this business by investing $200K of their own money into the business.

3. Suppose the churn rate is 15%, but it drops to 10% over time

4. Assume they also receive $5 per new customer as service revenue

5. In the beginning, we spent $5K/mo on research and development, which increases over time to $15K

6. we spend $5K/mo on marketing, which increases over time to $30K a month

7. Let's also assume that manufacturing costs increase over time from $500 to $2,500 (e.g., server costs).

8. And the G&A Costs increase from $3K to $9K over 2 years. 9. And the customer base grows from 10 new customers per month to 75 - this is of course, the most important and uncertain assumption, but we've to start somewhere.

After putting all the numbers into the model, we see that a pretty depressing picture emerges because in month 9, we've no more money in the bank - the MRR growth cannot sustain the burn, and there's still a lot of time before we break even. See the chart below.

This problem can be solved by raising additional funds. Let's say we raise another $300,000 in month 7, let's see how that changes. As we can see, this time it looks much better, at least we're not going bankrupt.

However, if we look at another chart that shows the remaining term, a risky picture emerges, because we'll have to deal with a term of one month for quite a while.

To avoid this risk, it might make sense to raise another $100,000 at the end of the first year, which works well in this case and allows us to safely cross the profitability threshold.

At the same time, in order to raise money in the future, we need to show the expected level of metrics.

The Gross margin seems pretty healthy if we can hit those numbers.

CAC Payback is high at the beginning but then normalizes to a decent value, well below the recommended 12 months.

MoM revenue growth also looks pretty healthy at around 10% per month.

This time looks like it's working! Now comes the most challenging part - the actual implementation. In this case, the biggest challenge is ensuring we can achieve this growth rate with the assumed marketing spend. And, of course, we can maintain the churn rate, find a product-market fit, save money, and so on...

This tool is helpful for checking your ideas and assumptions about a future business, but of course, it's a simplification, and for each case, you need to add more specific details.

If you're relatively new to SaaS, I will encourage you to play with the numbers to better understand how much churn, R&D spend, and other assumptions impact results.

You're welcome :)