We have all seen this movie while building an agent.

You begin with a question, refine the prompt, add another instruction, and try again. The answers improve until the system appears capable of completing useful work. It may still be rough, but it has moved beyond a demo and started to resemble an agent.
Then you let it run.

You change the prompt here, add a tool there, and introduce a skill somewhere in the middle. After a few iterations, the model is selecting tools, deciding how to sequence them, passing results between steps, and working out what to do when the expected output does not arrive.

I started calling this the LLM-as-orchestrator pattern.

It is often a very effective way to start the build because at the beginning of a project, you rarely understand the workflow well enough to define every branch, dependency, and failure mode. Letting the model attempt the task helps expose what the process actually requires.

The agent reveals missing tools, recurring decisions, awkward handoffs, and inputs that are less structured than everyone originally claimed, so you learn the workflow by watching an agent try to execute it.

The architecture, however, needs to evolve as that learning accumulates. Often, the agent begins following the same sequence across most runs, and much of its work becomes predictable execution. Keeping the LLM in control of every step then starts to add token cost, latency, and behavioral variance to a path the team already understands.

The useful production question is therefore fairly simple:

Which parts of the agent’s workflow have become ordinary software, and where does the code still run out of road?

And when that happens, what are the common signs, or smells, to look out for?

When the prompt starts looking suspiciously like code

You can usually see the transition by reading the prompt.

Early prompts describe an objective and provide enough context for the model to attempt the task. Later prompts contain increasingly precise instructions about which tool should run first, what fields should be extracted, how the output should be transformed, which conditions should trigger another action, and what should happen when a value falls outside an expected range.

The prompt gradually assumes responsibility for application logic.

At some point, it is worth asking whether much of that agent behavior would be clearer and safer in Python. Other languages remain available, apparently, although Python appears to have won the argument by refusing to leave.

This is where the phrase “prompt engineering” occasionally gives us more comfort than it deserves.

A prompt can contain substantial application logic while providing very few of the controls normally associated with software engineering: interfaces remain implicit and testing often focuses on the final answer rather than the route taken to produce it. A change in one instruction can affect behavior somewhere else, and the regression may remain hidden because the output still sounds plausible.

Plausibility is one of the less charming failure modes of language models. The system can produce a coherent answer after skipping a required check, choosing the wrong tool, or passing incomplete information downstream.

As the workflow becomes clearer, however, repeated behavior should start moving out of the prompt. That is when known transformations can become functions, stable payloads can become typed interfaces, and validation can move into schemas and assertions. At this point repeated sequences can be handled through deterministic orchestration and the model should remain where fixed rules, schemas, and ordinary control flow cannot resolve the case cleanly.

The accidental DAG

This transition normally happens one piece at a time.

A repeated instruction becomes a tool, a model-generated object becomes a structured payload, and one LLM call disappears because a rule can now produce the same result reliably.

Run it long enough, and eventually the traces show that the agent has called the same seven tools in the same order for the last twenty successful runs.

If you squint hard enough, this can resemble a self-improving loop.

It is, in a sense. You still have to inspect the traces, identify the pattern, extract it into code, test it, deploy it, and keep it alive while the system contributes mainly by generating the evidence and the invoice. Alternatively, you automate this with a “grade and trajectory analysis”, but in either case the agent is likely to converge into making ten tool calls back to back, passing the result of one into the next, before using a final model call to translate, classify, or summarize the output.

After that, you need to decide when the agent runs, so you add scheduling.
When it wakes up and there is nothing to process, you pay for the privilege of discovering that nothing happened.

When something important occurs between scheduled runs, processing is delayed.

When the workflow fails halfway through, you need state, retries, idempotency, and a way to recover from partial execution.

By this stage, the agent has known dependencies, defined stages, an execution schedule, and predictable failure modes.

You have accidentally converged on a directed acyclic graph.

Data and software engineers have spent decades solving this class of problem. Once the path is understood, conventional orchestration can execute it more cheaply and reliably, while giving the team clearer control over state, retries, monitoring, and deployment.

This is useful progress. The system has revealed enough of its structure to become ordinary software.

The only disappointment is that “scheduled workflow with a semantic transformation step” sounds considerably less magical than “autonomous agent”. Production systems have an unfortunate habit of losing some whimsy as they improve.

You can arrive from the other direction

You can also begin designing the workflow from the tools:
One deterministic function performs a defined operation. Another follows it.

The sequence is explicit, testable, and scheduled through conventional infrastructure.
You continue connecting steps until the path between two tools becomes difficult to express in code: perhaps the input is unstructured, or perhaps the next action depends on language that can reasonably be interpreted in several ways. Perhaps mapping one system into another would require a forest of brittle rules and regular expressions that nobody will want to maintain six months later (if you know Perl, I hear you may disagree).

That breakpoint is a natural place for an LLM.

The LLM receives the context, resolves the case, produces a structured result, and returns control to the deterministic workflow. It is being used because the software cannot cover the situation economically or reliably through fixed rules.

This route often produces a lighter agent because the LLM is invoked only where the normal runtime is insufficient while scheduling, retries, state, and observability are already handled by infrastructure designed for those responsibilities.

Agent-first and tool-first development therefore represent two different ways of discovering the same eventual boundary.

Agent-first development starts with broad flexibility and gradually extracts stable behavior. Tool-first development starts with a defined path and introduces the model at the points where the code is not enough.

Both can converge on the same production architecture.

The emerging architecture

The architecture I keep seeing emerge has three layers:

1. A deterministic backbone

2. An agentic exception path

3. A human escalation boundary

The deterministic backbone handles the normal path by owning stable sequencing, scheduling, validation, state management, retries, permissions, and observability.
The agentic exception path activates when the normal workflow reaches a case that the known rules cannot handle confidently. Here, “exception” covers more than a runtime error and includes any situation where the software lacks enough structure, context, or coverage to continue.

Consider, for example, all the cases where an input fails to match an existing schema, a document is incomplete, or two systems return conflicting information. And we could go on and on... a request may contain ambiguity that materially affects the next action, and a result may fall outside the range covered by the original rules.

In those “exception” cases, the model can inspect the context, call tools, gather additional evidence, and select a recovery path dynamically.

That is still agentic orchestration; it simply happens at the edge of the known workflow instead of supervising every routine step, and when the LLM resolves the issue, control returns to the deterministic path.

This approach paves the way for higher escalation. When the information remains insufficient or the action carries too much risk, the workflow escalates to a human in the loop.

The resulting three-step architecture gives the team a clear operating model where the known path is predictable, the exception path is adaptive and bounded, and the escalation boundary has a clear owner.

I quite like the approach as it is more useful than debating whether the system is sufficiently “agentic”: that term now covers everything from a loop around an LLM call to software with access to production systems and corporate credit cards. Some precision feels overdue.

Agent-first and tool-first solve different learning problems

Agent-first development works well when the workflow remains uncertain.
The model can attempt the task, expose missing tools, reveal hidden dependencies, and show which decisions actually recur. This reduces the cost of learning because the full process does not need to be formalized before anyone knows whether it works.
Its main risk is inertia: once the prototype begins producing useful results, delivery pressure rewards adding another instruction. The next stakeholder demonstration is approaching, someone needs the feature by Thursday, and restructuring the workflow feels slower than making the prompt slightly longer.

Over time, the prompt becomes the permanent runtime and the token cost, latency, and variance of the experimentation phase become production characteristics because nobody owns the industrialization.

Tool-first development works well when the execution path is already understood, or when reliability, auditability, and control matter from the beginning. The team implements the known workflow and introduces model calls where the software cannot cover the case cleanly.

That route has its own risk: teams can formalize the process too early and spend weeks implementing rules for a workflow they barely understand while several agent-driven attempts may have exposed the real structure much faster.

The starting point should reflect how much the team already knows and the final architecture should reflect how much uncertainty remains after the workflow has been exercised.

Four operating gates for every model call

When reviewing an agent workflow, I find these four questions useful.

Is the execution path predictable?

Look at the traces.

When similar inputs consistently produce the same tools in the same order, the agent is executing a known path and that sequence is ready to be considered for deterministic orchestration.

The length of the prompt is only a clue, while the agent’s behavior is stronger evidence. The useful questions are how often the path repeats, where it branches, and which steps continue to vary meaningfully.

Can the result be checked mechanically?

When an output must match a schema, satisfy a rule, reconcile against a known value, or pass an assertion, software should perform that check directly.

Asking a second model whether the first model followed a fixed structural rule adds another probabilistic step to a question with a deterministic answer, although it remains a surprisingly popular design, perhaps because one uncertain opinion feels lonely.

Model-based evaluation can still help when quality depends on meaning, completeness, or contextual relevance. Nonetheless, mechanical correctness should remain mechanical.

Is code enough to resolve the case?

Some steps are easy to express through fixed rules, typed interfaces, and normal control flow while others become expensive or brittle when forced into deterministic logic.

A model call earns its place when the software cannot cover the case cleanly, such as when it involves unstructured input, incomplete context, conflicting evidence, unfamiliar formats, or a large number of possible variations.

Moving data between known systems, applying fixed transformations, checking explicit conditions, and following a stable sequence rarely need a model. Token cost is only one part of the calculation, as even an inexpensive call introduces latency, variance, another failure mode, and another component that needs to be observed.
The LLM call should contribute something the runtime cannot provide more effectively.

Is the agentic path bounded?

Once the model can choose tools dynamically, the boundaries need to be explicit.
Which tools may it call? What data can it access? Which actions can it take without approval? How is recovery validated? When does control return to the deterministic path? Which conditions require human review?

These boundaries make agentic orchestration operable, and very importantly, make the system easier to explain to security, compliance, operations, and customers because the known path is predictable, the adaptive path is constrained, and the escalation point has an owner.

Without those controls, “agentic exception handling” can become a sophisticated phrase for granting broad permissions and hoping the traces are educational.

The organizational failure usually happens gradually

Most teams do not make a deliberate decision to keep an LLM responsible for predictable orchestration forever; they arrive there through a series of reasonable local decisions.

The prototype works → A customer asks for another case → A tool is added → The prompt receives another instruction → The deadline moves closer, and nobody owns the moment when stable behavior should be extracted into code!

At the speed of prototyping, each change makes sense on its own. Put together and left unchecked, however, those changes can create a production architecture whose cost and reliability remain tied to the way the team experimented.

Engineering leaders need an explicit review point for this transition. That review should examine path stability, token cost per successful run, latency, validation coverage, exception frequency, recovery behavior, and the proportion of model calls handling cases that software cannot yet cover.

The purpose is simple: as the team understands more of the workflow, the architecture should reflect that understanding.

Without this review, the agent remains the orchestrator by default. Stable behavior stays buried in prompts, and the organization continues paying for flexibility it no longer uses.

Let the architecture learn with you

Broad agentic orchestration can be a very effective way to explore a workflow early.
As the system runs, repeated patterns emerge. Known sequences can move into code. Common decisions can become explicit rules. Stable outputs can gain mechanical validation. Scheduling, state, retries, and observability can move into the runtime designed to handle them.

The agent remains available for the cases where code is no longer enough. It can handle unfamiliar inputs, reconcile conflicting information, coordinate recovery, and return the workflow to the known path.

That gives you the flexibility of an agent without charging the agent tax on every ordinary execution.

Use the model to discover the workflow. Move the parts you understand into code. Preserve agentic orchestration for the points where fixed rules and conventional software can no longer resolve the case efficiently.

Or, in terms my cloud bill can understand:

Thou shalt not spend tokens on software-engineering problems.

For years, we could manage that gap through spreadsheets because build effort still represented a meaningful share of the delivery timeline. Agentic development changed that balance dramatically and exposed how wide the gap could become.

With agentic development, an integration that once took weeks can now be built in a day, while its path to production remains largely unchanged. The result is a widening gap between what a team can produce and what an enterprise can safely adopt.

This creates a new leadership problem: work can look complete while remaining blocked by the conditions required to operate it. Projects can appear active without moving materially closer to production, and engineers can finish the build while the delivery state remains unresolved.

We quickly found that traditional measures of progress had become less reliable. They were designed for a world in which development effort and delivery progress moved together closely enough for one to serve as a proxy for the other. We needed a way to make delivery readiness visible, evidenced, and reusable across projects so that each project could make the next one easier.

FieldOne is the system we built to do that.

The spreadsheet worked until build time stopped being the constraint

FieldOne started as a spreadsheet that tracked live projects, effort, owners, dates, blockers, and slippage, which made the traditional delivery model visible and gave us a single place to plan the work.

The spreadsheet could show that an engineer was allocated to a project, but it could not capture the widening gap between compressed build time and the realities of enterprise deployment. Most importantly, a spreadsheet could not show the significant opportunity that the new way of working was presenting: engineers could now complete a build, switch context, support another project, or feed work back into product while an integration sat waiting on access, validation, or a customer environment.

Agentic development changed what we needed to see: the amount of code being produced no longer told us how much delivery progress had actually been made.

For a growing field engineering team operating across more concurrent projects, the spreadsheet could no longer represent the work. We needed a shared state that showed what was active, what was ready, what was blocked, and where agentic speed was creating leverage or risk. If the original spreadsheet tracked hours, the next system needed to track delivery truth.

The spreadsheet showed us what the operating model was missing

The second iteration of FieldOne was still a spreadsheet, but it gave us a more useful view of delivery by plotting milestones over time and layering in the skills, owners, and dependencies required at each stage. What looked like a planning tool soon exposed the operating shape of the problem: projects were no longer moving in step with engineering effort, but through periods of build, waiting, validation, handoff, and escalation, often with responsibility shifting between field, product, the customer, and external system owners along the way.

That view made readiness easier to locate. We could see where a project had stopped needing active engineering but was still unable to advance, where the next dependency sat outside the person formally assigned to the work, and where delays were accumulating without appearing in the capacity plan. Access, environments, permissions, data ownership, validation, security, UAT, and operational readiness were shared delivery conditions, and when they were not surfaced early, they returned later as rework, delay, and commercial risk.

As the model became more useful, the spreadsheet became less able to contain it. The relationships between projects, milestones, owners, dependencies, evidence, and risk were becoming too complex for rows and formulas, while the reporting and workflow logic increasingly belonged in a system rather than a shared file. So we moved FieldOne into a dedicated application with its own database, backend, user experience, reporting layer, and workflow model, all built on our own product layer, ContextOne.

That transition turned FieldOne from a planning aid into the system of record for delivery state, with structured objects and lifecycles for projects, phases, ownership, evidence, risk, and readiness. ContextOne supplied the integration, automation, and agent capabilities around it. Delivery conditions became visible, repeatable, and governable without requiring agents or operators to reconstruct the truth from scattered updates.

Once delivery existed as a structured state rather than rows in a spreadsheet, we could begin designing the operating model around what the data was showing us. That model settled into four practices: classify work by delivery archetype, promote repeated work into certified capability, make delivery health evidence-based, and make the reliable path the default.

1) Classify work by delivery archetype

The first pattern FieldOne made clear was that projects differed less by size than by the shape of the work required to move them forward. No two projects were the same, but they rhymed. Most fell into three broad archetypes, each with a different mix of skills, dependencies, and operating risk.

Agent-heavy projects reused a significant amount of capability from ContextOne and required relatively light last-mile configuration or integration. Their progress depended less on new infrastructure and more on subject-matter expertise, workflow validation, and clear acceptance criteria. The technical work often involved configuring an agent loop around a known set of tools, behaviors, and validation steps, while the harder part was confirming that the workflow reflected how the customer actually operated.

Integration-heavy projects connected fragmented data and processes across multiple systems. They required more field customization and more coordination across system owners, permission models, data structures, and reconciliation logic. They also generated some of the most valuable product learning, because connectors and workflows that began as project-specific work often revealed patterns that could be hardened and reused elsewhere.

Build-heavy projects extended the boundaries of the product itself. These were first-of-a-kind deployments that required new capability, such as a fully air-gapped on-premise environment, a custom operating system kernel, a 1,000-user concurrency certification, or a customer-led UAT process supported by dashboards that operational teams could use without us in the room. They needed deeper involvement from product, engineering, and infrastructure because the delivery work was also creating part of the future product.

The archetypes helped us understand why projects with similar timelines could require completely different interventions. Agent-heavy work needed more subject-matter expert (SME) access and UAT coordination. Integration-heavy work needed stronger project management and cross-system ownership. Build-heavy work needed deeper product and architectural support.

Once those differences were explicit, we could design delivery pods around the actual shape of the work rather than staffing every project as though it followed the same path. The same model improved capacity planning, clarified where risk was likely to appear, and helped us distinguish between work that should remain customer-specific and work that warranted product investment.

2) Promote repeated work into certified capability

The next question was what should happen when the same integration, workflow, connector, or agent behavior appeared across several projects.

A working implementation was not enough to make something reusable. Production constraints such as environment parity, permission models, scale, rate limits, payload size, timeouts, monitoring, rollback, and operational ownership could still be untested, even when the feature worked correctly in development. Reusing that work without understanding those limits simply moved the uncertainty into the next project.

We created a promotion path that turned repeated project work into certified capability. To move into the product path, the work needed a named owner, supporting evidence, defined prerequisites, known operating limits, reuse criteria, and production-readiness checks. This gave the next team something they could depend on rather than a previous implementation they first had to reverse-engineer.

The purpose was to make learning portable. A project should leave behind more than code that once worked in a particular environment. The next project should inherit progress, not repay the zero-to-one cost.

Defining what made a capability reusable also forced us to define its production conditions earlier. Many of those conditions were knowable before kickoff, which changed how we approached pre-sales. We introduced a small set of questions covering the systems involved, the applicable permission model, the available environments, known technical limits, customer ownership, and the evidence that would count as production-ready in that operating context.

These questions did not add a new approval layer. They made assumptions visible before they became embedded in the plan, giving both sides a clearer view of the conditions required for delivery to proceed.

3) Make delivery health evidence-based

As the portfolio grew, project health became increasingly difficult to interpret because terms such as “on track,” “nearly complete,” and “green” often described different realities depending on who was speaking. One person might be referring to the state of the build, another to customer validation, and another simply to the absence of a newly reported blocker. Reviews therefore spent too much time reconciling competing versions of the project before the team could decide what required action.

We wanted those conversations to begin from a common set of facts: the dates and scope that had been committed, the current delivery phase, the evidence available, the age and ownership of open blockers, and the risks being carried into the next checkpoint. These questions became the basis of the Theta Dashboard, named after the options concept of time decay. In FieldOne, theta represented the delivery cost that accumulated as the committed plan and the current reality moved further apart.

FieldOne calculated carry, slippage, and trajectory for each phase by comparing planned dates and scope with the current forecast, evidence freshness, blocker age, active exceptions, ownership, and phase-gate status. The underlying fields remained deliberately simple, but together they made the delta visible early enough to act on. A project described as on track now needed an aligned forecast, current evidence, clear ownership, and an acceptable blocker and exception state, turning health from a summary judgment into a claim that could be inspected.

To support that model, we defined phase gates and checklists around the production conditions that mattered in practice. A gate was complete only when the required evidence existed or an explicit exception had been recorded. “Access verified,” for example, required an approval artifact or a successful test query linked to the milestone and a named owner, while “Environment parity verified” required evidence that the deployed configuration matched the production target. By tying sign-off to evidence, unresolved conditions became visible before they could pass silently into the next phase.

4) Make the reliable path the default

Evidence made risk visible, but it did not ensure that teams followed the reliable path once delivery came under pressure. Optional process disappears the moment urgency shows up, so the reliable path had to become the default path, built into the way delivery operated.

Deployment patterns, agent workflow templates, integration checklists, validation routines, rollback plans, and production-readiness criteria became first-class delivery assets rather than supporting documents scattered across folders and chat threads. Each pattern followed a consistent structure covering prerequisites, environment and permission checks, implementation steps, validation, rollback, and the criteria required for production use.

That structure turned individual experience into something another engineer could apply without first reconstructing the reasoning behind the previous deployment. A new field engineer could begin with a known pattern, confirm which prerequisites were already satisfied, and focus on the parts of the environment that were genuinely different. Customers and delivery teams benefited from the same clarity because both sides could see the established path, the decisions still outstanding, and the places where the deployment departed from the norm.

The model still allowed teams to move forward when reality required a deviation, but every bypass had to be recorded as an explicit exception with a rationale, owner, expiry date, available evidence, mitigation, and follow-up action. That preserved flexibility while turning silent risk into visible risk.

Over time, the exceptions themselves became a source of operational and product learning. When several projects bypassed the same gate for the same reason, the pattern pointed to a weak template, a missing readiness check, an unrealistic process, or a capability ContextOne needed to provide. Delivery friction could then be converted into a stronger default rather than rediscovered by the next team.

By this stage, FieldOne was no longer recording delivery after the fact. It was governing how work moved across field, product, agents, customers, and the systems they depended on.

What we learned

The most important lesson from building FieldOne was that managing delivery in an agentic organization requires more than adding automation to an existing process. Once agents began compressing build time, shifting engineering capacity between projects, and turning delivery work into reusable product capability, we needed a system that could represent the new operating model directly.

That meant owning the delivery object model first. Projects, phases, owners, evidence, risks, decisions, dependencies, staffing, product signals, and health all needed to exist as structured state with clear relationships and lifecycles. Without that foundation, automation would have accelerated the production of updates without improving our understanding of what was actually ready, blocked, or at risk.

The separation between delivery state and the automation around it made FieldOne both useful and governable. FieldOne remained responsible for the record of what had happened, what had been approved, what was waiting, and what needed to happen next, while ContextOne provided the agents, integrations, and workflows that could act on that state. Agents could prepare meeting briefs, surface aging risk, draft customer updates, or recommend next actions, but they were operating within an explicit delivery model rather than reconstructing one from scattered context.

We also learned that signals beat status. A weekly review should not begin with each owner explaining whether a project feels on track, but with changes to the forecast, aging blockers, missing evidence, active exceptions, phase-gate status, and ownership of the next action. This shifted reviews away from reconciling competing narratives and toward deciding where intervention was required.

The same principle applied to process. Optional practices tend to disappear under pressure, so gates, templates, readiness checks, and evidence requirements had to become part of the default path. Teams could still bypass a gate when delivery required it, but the exception had to be named, owned, justified, time-bounded, and tied to mitigation. Flexibility remained possible without allowing risk to become invisible.

Over time, those exceptions, escalations, and postmortems created a compounding loop. A repeated problem could become a stronger gate, a clearer readiness check, a certified capability, or a reusable delivery pattern. Each project could therefore improve not only its own outcome, but the system used to deliver the next one.

That is the larger role FieldOne came to play: managing delivery in a world where agents had changed the speed, allocation, and economics of the work.

As AI reduces the effort required to build, the scarce resource becomes trustworthy progress: work that survives contact with a real customer environment, can be inspected and governed, and leaves behind reusable capability rather than another isolated implementation.

The goal is not to make enterprise delivery simple. It is to ensure every project makes the next one easier.

More often than not, the failure is not particularly dramatic. Two sessions receive the same request, use the same model, and appear to call the same tools. One completes. The other takes a slightly different path, nothing dramatic, retrieves the same records in another order, retries a tool after a timeout and arrives at a completely different answer. Both traces look plausible. Neither explains why they diverged.

This is the point at which an agent platform stops behaving like traditional software and starts behaving like several nearby versions of the same software, selected by timing.

It’s not the model

The ‘traditional’ response to this is to investigate the model. Temperature is reduced, prompts are tightened, perhaps a stronger model is substituted. These changes can improve answer quality, but they do little for execution consistency because the model is rarely the only source of variation.

A production run is shaped by the order in which messages arrive, which tool result returns first, whether a request hits a cache, how a retry is scheduled, which retrieval candidate receives a marginally higher score, and what state was left behind by the previous turn. Even timestamps and generated identifiers can alter later behavior if they enter the context or are used as lookup keys.

None of these variations is large enough to look like a platform fault on its own, but together they destroy reproducibility.

So does that matter? Well, yes. Reproducibility is the boundary between an operational system and a convincing demonstration. A demonstration needs to work, perhaps just to justify a budget. An operational system needs to explain what it did, resume after failure, and produce evidence that a proposed fix addresses the same execution that failed.

In short - a log or a trace is not enough.

Logs are observations emitted by a running process. They are frequently incomplete, asynchronously written, and detached from the state that gave an event its meaning. A tool log may show that a request was submitted twice without revealing whether the second submission reused the first execution, started a duplicate job, or observed different upstream data. A model trace may contain the final prompt while omitting the retrieval ordering, evicted tool output, or cache decision that produced it.

The Missing Layer: Deterministic Execution

The missing component to all this is a deterministic execution layer between the model and the distributed systems around it.

That layer does not necessarily attempt to make language generation mathematically deterministic. What it does do is make the surrounding computation deterministic enough that model variation is visible rather than mixed with orchestration noise.

Strict Event Ordering

The first requirement is strict event ordering.

Agent runtimes are naturally exposed to concurrency. User messages arrive while tools are running, and multiple tools may be dispatched in parallel. Provider streams emit incremental output. Recovery policies introduce retries, route changes, and context compaction. If these events mutate session state as they arrive, wall-clock timing will naturally become part of the program.

A hundred real world variations can cause this to happen. A tool that completes 20 milliseconds earlier may enter context first. A retry callback may race with a cancellation. A session update may be accepted between two tool results. The resulting trace will still look reasonable, but the state transition order is no longer stable.

A more reliable runtime treats inbound activity as an ordered event stream. Events may be produced concurrently, but state changes are applied synchronously by one authority. The core consumes one ordered inbox and advances the session through explicit transitions. Model requests, tool completions, retry decisions, recovery actions, and external inputs all acquire a defined position in that sequence.

This design can sound conservative to teams accustomed to extracting parallelism from every service, but crucially, the restriction applies to state mutation, not to all work. Tool calls can still execute concurrently. Retrieval can still fan out. Model traffic can still be streamed. What cannot remain ambiguous is the order in which their results become part of the agent’s state.

A single-threaded deterministic core is often the simplest implementation. It removes an entire class of locking, race, and causal-ordering problems. More elaborate state machines can provide the same property, but they still need a single agreed sequence for state-changing events and a replayable transition function.

Timestamps and identifiers belong inside this boundary as well. If they are generated from ambient wall time or random UUIDs during replay, the replay is already different. A deterministic clock and reproducible identifier generation allow restored execution to recreate the same internal references and event metadata.

Materialized Execution Snapshots

This ordered core becomes the foundation for the next requirement: materialized execution snapshots.

Agent sessions are commonly persisted as conversation transcripts. That captures what the user and model said, but not the execution environment in which the conversation occurred. The transcript may refer to files that have since changed, tool results that were truncated, retrieval outputs that were never stored, or subagent state that existed only in memory.

A production checkpoint needs to include the state the agent could actually act upon.

That means sealing the core session state together with the execution trajectory and the session’s working environment. If the agent can create files, transform data, spill large tool outputs to storage, or continue a child agent later, those artefacts are part of the session. Persisting the messages without them is comparable to storing a database transaction log while discarding the database pages.

A practical checkpoint can be taken at the end of each turn, when the runtime has reached a durable boundary. The snapshot should contain the model-neutral conversation representation, active policies, tool and recovery state, session labels, durable child conversations, the virtual file system, and a complete trajectory of decisions and external results.

The archive can then be sealed for integrity and written to object storage as one recoverable unit. A compatible runtime should be able to load it and continue from the last committed turn without reconstructing the session from scattered service logs. Before resumption, the target runtime should verify that the required tools, model capabilities, and configuration contract are still available. Why? Because silent restoration into a materially different environment is another form of nondeterminism.

Once snapshots exist, context construction must be derived from them rather than rebuilt opportunistically from live systems, and this is where many otherwise careful platforms lose determinism. A replay loads the same transcript, then reruns retrieval against a changed index. A tool catalogue is searched again and returns a slightly different ranking. Context compaction chooses a different boundary because token estimates changed. An old tool result is re-fetched from a system whose data has moved on.

Instead of a replay event asking “What context would the system build now?”, a better question is “What context did this state produce then?”

Context policy can still be dynamic of course. Large results may be moved from the prompt into a session file system. Older turns might be compacted. Tool responses may be re-encoded into a more token-efficient representation. The important property is that these actions are explicit state transitions recorded in the trajectory, and that these transitions are accurately reflected in the snapshot.

The same discipline applies to retrieval. Search systems routinely produce ordering variance because of index updates, floating-point scoring, replica differences, or ties resolved by execution timing. A production agent should either materialize the selected retrieval set as an event or query against a versioned, time-bounded view that can answer as the data existed at the time of the decision.

Without all of this, “replay” means submitting a similar request to a different world.

Tools as State-Bound Events

Tool execution is the most consequential boundary because tools create side effects.

Many orchestration frameworks treat a tool call as an ordinary function invocation. The runtime sends arguments, waits, and inserts the result into the transcript. This abstraction conceals the distributed transaction underneath. The request may be retried by the client, the service mesh, the gateway, or the tool provider. A timeout may mean that the operation failed, or that it succeeded and the response was lost.

A deterministic runtime must therefore treat a tool invocation as a state-bound event with a stable identity. The call must carry an idempotency key derived from the session, turn, and tool event. A retry must resolve to the original execution wherever the tool contract permits it. It must not create a new side effect merely because transport delivery was uncertain.

The gateway can remain asynchronous internally while presenting a synchronous result to the agent. It may submit a job, poll its status, and return only when the job succeeds, fails, or times out. The agent does not need to reason about the gateway’s scheduling. The runtime does need to preserve the job identity, dispatch decision, arguments, result, and retry relationship.

The rule is simple even when the implementation is not: uncertainty about the response must not become uncertainty about whether the side effect occurred.

Mutable storage needs the same treatment. Agent file writes should be versioned and protected by optimistic concurrency. A caller that edits a file must identify the version it observed. If the head has changed, the write should fail and force a re-read rather than silently overwriting a concurrent update. Each successful mutation should create an immutable version linked to its predecessor and attributed to the source session and tool event.

This will produce more than simple auditability. It will ensure state drift is visible at the point where it occurs.

An Incremental Implementation Path

For in-house builds, the implementation path is usually incremental. The ordered event core should come first because every later guarantee depends on it. Existing tools can initially be wrapped as events even if their downstream idempotency remains imperfect. Turn-boundary snapshots can then capture conversation state, event history, and tool outputs. The session file system and other working state can be added once the restore contract is stable. Context policies can subsequently be moved into deterministic transitions, followed by stronger tool deduplication and versioned external state.

Trying to begin with a complete reference architecture tends to create a large program with little early operational value. The useful milestone is narrower: take one failed production session, restore it from a committed boundary, and reproduce the same state transitions without contacting systems that were not contacted in the original run.

The architecture we describe here evolved over a focused period of real world implementations, culminating in the “ContextOne” product. We use an ordered, single-threaded core, deterministic clocks and identifiers, turn-boundary archives containing sealed session and file-system snapshots, durable trajectories, idempotent tool identities, version-aware writes, and recovery policies owned by the core rather than hidden inside transport layers. For the curious, single threaded does not mean slow either - in a 30 minute long agent turn, our own overhead is less than 10 milliseconds.

These choices impose cost. More state is persisted. Tool results must be retained. Event schemas need versioning. Compatibility checks become part of deployment. Engineers must distinguish between parallel execution and ordered state application. Product teams lose some freedom to patch behavior through invisible middleware.

The return is not simply fewer failures. It is control over failures.

When a model produces a different answer from the same preserved state, that difference can be measured as model behavior. When a tool returns different data, the changed external observation is visible as an event. When a retry occurs, the system can show whether it reused or duplicated execution. When a deployment changes prompts, tools, or recovery policy, recorded sessions can be replayed against the candidate configuration and checked for regressions.

That is the operational value of determinism in agent systems. It turns an opaque sequence of plausible actions into a state machine that can be recovered, audited and compared.

Senior engineering leaders do not need agent systems to behave identically forever. They need to know which differences were intentional, which were produced by the model, which came from changing data, and which were introduced by the runtime itself. Without a deterministic layer, those causes remain entangled. Every incident becomes a reconstruction exercise, and every apparent fix is tested on a run that may not be the one that failed.

Production trust begins when the system can preserve its own history closely enough to disagree with it.

Climbers have a name for that: the false summit. You crest what is plainly the peak, and the real one appears behind it, higher, colder, and further off than anything you have climbed. Enterprise AI is a mountain made almost entirely of these, and the cost of mistaking one for the top is measured in quarters, not days. A decade ago there were a few hundred AI companies; today there are tens of thousands, and every one of them is standing on one of these ridges, telling you it is the top.

Two ridges come before the real climb even starts, and nearly everyone clears them and calls it progress.

The first is the demo. An agent answers a hard question against a clean, hand-picked set of sources, in a sandbox, in front of the executive committee. It is convincing precisely because everything that makes the enterprise hard has been removed: the data is tidy, the questions are the ones the demo was built for, and nobody is asking what happens on Monday when a thousand people use it. A demo proves something is possible. It does not prove you can deliver it.

The second is enterprise chat. You switch on an assistant for everyone, wire it to email and documents, and the adoption charts climb. The gains are real, but they are small, and they show up in the employee’s day, not on the income statement. I have started calling this corporate wellness: a decent thing to offer, and people feel supported, but it was never going to change the financials. Set a usage target like “use the assistant four times a day”, and people will ask about the weather to hit the number.

Both ridges make the same mistake, and it is the one hiding under every false summit higher up: they confuse the how for the why. Across the 477 of these conversations my team has sat in since January, everyone wants the same thing, and it is not AI. They want operational leverage, the ability to do more with what they have. AI is the how; operational leverage is the why.

It helps to be precise about the work AI should actually do. The sweet spot is high complexity, low judgment tasks. High complexity, because if the task were simple you would have automated it years ago with if-this-then-that rules. Low judgment, because you do not want a model deciding which drug to prescribe or whether to approve a mortgage. You want it to do the gathering and cross-checking, the swivel-chair work, and hand a complete picture to the expert who makes the call. Keep that test in mind and the real false summits explain themselves.

Above the tree line, the climb gets serious. These are the four that cost real years.

Summit one: ten times the code

The first real false summit is AI-assisted software development, and it is seductive because the gain is visible: the tools work, and engineers produce dramatically more code. The mistake is reading code velocity as product velocity.

Across the teams I see, 10x the code is not turning into 10x the product, or even a noticeably faster roadmap. One CIO told me an AI agent had rebuilt the same component 76 times in a month, a component that already existed, leaving him with 76 versions to maintain and a strong wish to put the genie back in the bottle. He has since taken the tools off his developers, not because they do not work, but because the old way of specifying, reviewing, and shipping software cannot absorb that volume. Writing the code was never the bottleneck. The real constraints were deciding what to build, checking it, and integrating it safely, and those are the parts the tools do not yet speed up.

Code was never the constraint. Until the process around it is redesigned, more code just means more to review.

Summit two: centralizing the data

Once teams build agents that act rather than answer, they hit the real state of enterprise data: scattered across dozens of systems, structured and unstructured, some of it true, some stale, much of it contradictory. The instinct is the one the industry reaches for every cycle: centralize. Get everything into one place so the agent has one surface instead of forty. Whole programs are commissioned on that theory, often with a multi-year price tag.

Then you finish the migration and find that location was never the hard part. The agent can reach everything and still does not know what anything means. Roughly a third of the meaning is clear. Another third is ambiguous: is the “customer” in one system the same as the “customer” in another? The final third was never written down at all; it lives in the heads of people who have done the job for fifteen years. Moving data recovers none of that. The first third is software, the second needs experts resolving cases one at a time, and the third can only be learned by watching how the work is actually done.

There is a second cost that rarely gets priced. When you centralize, you hand your hardest-won knowledge to whichever platform you migrated into. For a one-off report, fine. For the understanding that makes your business run, it is worth asking whether you want it living inside a single vendor.

The battle is over meaning, not location. Pay for understanding before you pay for migration.

Summit three: the first dozen agents

This is the summit that fools the most capable teams, because by the time they reach it they have done everything right and have the scars to prove it. I had dinner recently with senior AI leads from some of the largest financial institutions in the world. The pattern around the table was remarkably consistent: a dozen, maybe two dozen, meaningful agentic experiences in production. Built by dedicated teams that started around fifteen people and grew to thirty-five over the twelve months it took.

Now do the arithmetic out loud, which almost nobody does. A win is not a dozen agents; inside a large enterprise the real surface area is thousands, maybe tens of thousands, of these workflows. If the first dozen took a year and a team of thirty-five, the current approach gets you to full scale somewhere around 2035. Every leader I put that number to goes quiet, because they have done the same math privately and kept it to themselves.

The reason sits in how those first agents were built. Getting an answer at all is the hard 30%, the context work, wired by hand for each agent. Earning the right to trust the answer is the other 70%: the evaluations and checks, the machinery that catches a mistake before a customer does. Almost all of it is hand-built, per agent. The thirteenth agent costs roughly what the first one cost. Nothing accumulates. That is the defining feature of a false summit: the effort does not turn into height.

Track the cost of your next agent, not the count of your last ten. A flat cost curve at agent twelve is your next five years.

Summit four: the bill

The last summit arrives as a reward for success. One bank built an internal assistant for its developers and tested it on five small systems, where it was accurate and cheap. Then they pointed it at three dozen real ones. The answers held up; the cost did not. Queries that mattered now ran past two dollars each across tens of thousands of engineers. The options were all bad: move everything into a warehouse, pay the token bill and hope, or rebuild the data a third time as a graph. None of those is a strategy.

This is the part nobody prices in during the pilot. Metered tokens look harmless at prototype volumes. At enterprise scale, with agents making dozens or hundreds of model calls per task, the bill starts to behave like a cloud bill: opaque until you instrument it, volatile until you govern it, and politically radioactive once it shows up in operating expense. CFOs have seen this film before, when predictable capital spend turned into unpredictable cloud spend, and they remember the ending. Two things make it worse. The same task that costs ten dollars on a frontier model is often a fifty-cent task on a smaller one, but only if your setup can route between models at all. And the “best” model keeps changing, so anything hardwired to a single provider is a depreciating asset.

What matters is cost per finished piece of work at a set quality, and the ability to swap the model underneath it in weeks, not quarters.

The actual mountain

So what does the real summit look like? It is a system that actually holds up in real-world conditions: answers it can trust and stand behind, on the messy data it already has, fast and affordable across the whole enterprise. Every false summit falls short of that in a different way. The coding wave produced far more code, but not more finished products. Centralizing the data put everything in one place, but the agent still could not tell what any of it meant. The first hand-built agents worked, but each new one cost about as much as the last: the cost never fell and nothing carried over, so the approach could never scale. And the moment real usage arrived, the token bill made it impossible to sustain.

Underneath all of them is one shift most enterprises have not absorbed: this is the first major wave of technology built on probabilistic foundations. Every previous system gave the same answer to the same question every time. A language model does not. Ask it the same question five times and you can get five different answers, each delivered with full confidence. Everything an enterprise means by “production” was defined in a deterministic world: auditability, repeatability, accountability. Crossing that gap is the whole climb.

What the crossing demands turns out to be specific, and it is not more clever agents. It is three disciplines.

The first is a living data foundation, built where the data already sits. Not another migration into a warehouse, but software that reads every source in place, whatever the system or data model, structured and unstructured alike. It works out which entities are which, fills in the missing context, and keeps re-resolving as schemas, joins, and meaning shift underneath it. A static catalog is stale the moment it is finished, because the business does not hold still. A foundation that re-resolves against live sources and real workflows is the thing centralizing was reaching for and never caught.

The second is context and guardrails built by software, not by hand. The 30% that surfaces an answer and the 70% that earns the right to trust it: the evaluations, the judges, the checks, generated and enforced across a whole population of agents at once, with evaluation kept permanently in the loop. Crafting one agent by hand is tractable. Governing thousands of them, as models and prompts drift beneath you, is not. Not by people, not ever. It is the only thing that makes effort accumulate into altitude instead of starting again from zero at agent thirteen.

The third is the move from exploration to dependable execution. Let an agent reason a task out once, in full; grade that path; and when it is proven, promote it to plain code that runs the same way every time, with the model verifying the result rather than rediscovering it. The same question stops returning different answers. The cost stops rising every time you ask. Proven work hardens into infrastructure, which is exactly what a regulated business means by production.

Between them, those three clear the false summits that cost companies years: the meaning, the hand-built dozen, and the bill. None of it looks like building impressive individual agents. It looks like building the thing that builds them: the shared understanding of the business, the machinery that checks the work, and the path from probabilistic exploration to deterministic execution. The agents are the easy part. The discipline underneath is the climb, and it is where the advantage that compounds for a decade actually lives.

Two principles hold it together. Start with the AI working in the background while a person signs off on every result, and let it earn more autonomy only on evidence, not enthusiasm. And design accountability in from the first day, because when an AI-assisted decision goes wrong, “the system told me” is not an answer.

None of this is a counsel of despair. The mountain is climbable, and the teams that climb it will hold an advantage that compounds for years. But the route matters more than the enthusiasm. So one question to carry into your next AI review: are we higher than we were last quarter, or are we standing on another ridge? The teams that ask it early save themselves years.

I agree with the sentiment. The job is increasingly to write loops. But the more important thing is what a loop actually is.

A loop is not simply “let the model run for a while.” In programming, a loop has structure. It says: while this condition is not true, do this thing. Or: continue until this condition is met. There is always an evaluation followed by an execution. The system checks the state of the world, decides whether the condition has been satisfied, and either exits or performs another action. That is what makes it a loop rather than a long prompt.

So the real work is not merely creating an agent that can keep acting. It is defining the evaluation that tells the agent whether the last action mattered. What did we put inside the parentheses? What counts as done? What counts as better? What counts as a regression? What state needs to be carried forward so the next iteration is more informed than the last one?

That is why I think “loop engineering” is directionally right but incomplete. The leverage is in the evaluation. We are not just writing loops; we are creating systems that can evaluate chunks of work, decide whether they satisfy a condition, and then execute the next step accordingly.

Without that evaluation, a loop is just automated continuation. A powerful model can continue for a very long time. It can generate plans, search through tools, inspect files, make edits, summarize its progress, and try again. If the original goal is vague enough, it may eventually produce something plausible. But plausibility is not the same as correctness. If we say “build me App X,” and provide little else, then the model has to infer what App X means. It has to infer the users, the data model, the workflows, the edge cases, the integrations, the permissions, the interface, and the definition of success. A better model and a larger token budget will improve that process, but they do not eliminate the basic problem. The loop is spending intelligence to compensate for the absence of evaluation.

Software engineering gives us the cleanest version of this idea because software already has a culture of evaluation. A code agent can write code, run tests, inspect failures, patch the implementation, and repeat. In that environment, the loop has something concrete to check. Did the test pass? Did the build complete? Did the type checker succeed? Did the output match the expected fixture? The agent does not need to guess forever whether it is making progress. It can be told.

This is also how we think about our own agent infrastructure. One of the most important things we built was not a more elaborate prompt, but a deterministic way to evaluate the harness around the model. By using a mock LLM interface, we can test tool calling behavior, component interaction, handoffs, and context transitions without relying on a live model response every time. That lets us test the system around the non-deterministic component. Once that structure exists, the evaluation surface expands. We can run throughput tests, send millions of messages through the system, watch context compact and rehydrate, and observe agents operating as long-running processes rather than isolated chat sessions.

The lesson is not that everything should be deterministic. The lesson is that the non-deterministic part needs a deterministic frame around it. The model may decide how to interpret an ambiguous situation, but the system should know what state it is in, what tools are available, what result is expected, what action was taken, and what should be evaluated next. That is the difference between an agent that is merely active and an agent that is participating in a controlled process.

This becomes more important when we move from software engineering into enterprise workflows. Codebases come with files, tests, logs, types, git history, and build systems. Enterprises have SaaS tools, spreadsheets, process documents, ticket histories, Slack/Teams threads, data warehouses, browser-only workflows, tribal knowledge, and business rules that live in people’s heads. The process may be obvious to the team that runs it every day, but invisible to an agent unless it has been represented somewhere.

Take a simple business task: review a situation, decide what should happen next, and route it through the right process. On the surface, that sounds small. In reality, it depends on business context that is rarely contained in the prompt. The agent needs to know what the relevant entities are, where the data lives, which systems are authoritative, what the organization means by status or risk or priority, what actions are allowed, and what outcome would count as correct. A human employee often carries that context implicitly. They know which dashboard matters, which field is stale, which exception is normal, and which process doc is out of date. An agent does not know any of that unless the organization has represented it somewhere.

A high-end model can help discover these things. Given enough access and enough budget, it can inspect systems, read documentation, query endpoints, observe patterns, ask questions, and assemble a plausible ontology of the business. That is incredibly valuable. But if every future loop has to rediscover that ontology from scratch, then we have not built a system. We have built an expensive ritual.

The better pattern is to spend intelligence on discovery once, then preserve the result as managed context. This is how we think about the Agent Ontology Service, or AOS. A capable model can be given read-only access to explore the relevant systems, validate endpoints, map entities, identify relationships, and build an ontology layer around a business process. It can connect the language people use in the business to the systems where that language becomes data.

Once that ontology exists, the loop changes. The agent no longer has to begin with “go figure out what this company means.” It can begin with a more precise instruction: execute this process against this known context, under these constraints, and evaluate these outcomes. That shift matters enormously. It changes the cost profile because the loop is not spending tokens rediscovering the same background reality. It changes the reliability profile because the loop is now operating against a shared representation of the business. And it changes the improvement curve because the evaluation layer can become more complete over time.

This is the enterprise version of evaluation-driven development. Every business process contains a mix of deterministic and non-deterministic work. The judgment calls will remain judgment calls. The model may need to classify a situation, infer intent, summarize evidence, recommend an action, or decide which precedent applies. But once that ambiguous step has been resolved, the surrounding process can often become deterministic. If a risk is identified, route it to the right owner, attach the evidence, update the relevant system, create the follow-up task, and log the rationale. If required data is missing, request it from the source of truth. If an exception violates policy, escalate it through the correct path. If the workflow completes, write back the outcome and update the context for the next loop.

That is where context management becomes the control plane for agentic work. Context is not just memory. It is the durable state that allows loops to evaluate correctly across time. It tells the agent what exists, what matters, what has already been tried, what constraints apply, and what success looks like. It turns discovery into an asset rather than an expense. It allows expensive, high-autonomy work to be distilled into lower-cost, higher-determinism execution.

So yes, the job is to write loops. But in the enterprise, the job is more specifically to design the evaluations that make those loops worth running. The loop is the structure: evaluate, execute, repeat. The evaluation is the leverage. Context is what lets that learning persist.

The teams that win with agents will be the teams that know how to define success, capture business context, preserve ontology, and convert repeated judgment into durable process. The next frontier is not agents that can keep going. It is systems that know what they are checking for when they do.

By lunch, it has done this a dozen times. The team may call that proactive AI. The finance system will call it compute. The platform team will call it load. The account manager will ignore the notifications because most of them say nothing.

Then, at 14:37, the thing that matters happens. A support ticket from a renewal account is raised to severity one. Usage has already dropped. The executive sponsor changed jobs two weeks ago. Legal has an unresolved contract question. The event is sitting there in the business systems, crisp and timestamped.

The agent still waits for the next scheduled run.

When it finally wakes, it has to rediscover why the account matters. It loads too much context, misses the contract issue, pings the wrong owner, and drafts a generic account note that the customer-success lead quietly rewrites. Essentially, the save motion has started too late. The first internal escalation is noisy. The customer receives a thin response precisely when they need reassurance that someone truly understands their needs. Nobody built a broken model here, they built the wrong waiting architecture.

At OpenAI’s Intelligence at Work livestream, Sam Altman pointed to “constantly running proactive AI” as the next major phase after chat models and agent-style systems such as Codex. His larger point was that companies will have to change how they implement AI, starting now.

That implementation change is where things get interesting. If enterprises hear “constantly running” and respond by scheduling more model calls, they will recreate the oldest mistake in distributed systems with the most expensive component in the stack.

The architecture question has moved: what event gives the agent a reason to think?

That is the first question I would ask of any proactive AI design. If the answer is “a timer,” the system may still be useful. It can produce a Friday report, check a known queue, or send a recurring reminder. A timer proves that time passed. It says nothing about whether a business event occurred.

A serious proactive system starts with the event. A complaint crosses a severity threshold. Inventory falls below a buffer. A payment fails. A fraud score jumps. A supplier misses a milestone. A policy changes. A VIP customer goes quiet after a bad onboarding call. These are deterministic facts before they are reasoning problems.

Those signals rarely arrive ready for an agent. The runtime has to turn the raw change into a wake decision: resolve the business object, enrich the event just enough to route it, check the agent registry, and decide whether any hibernating agent should resume. Sometimes the answer should be no. Sometimes the event should be stored against an agent’s state and wait for a second signal. Sometimes it should wake the agent immediately because the next useful move requires judgment, language, or action.

That is the practical meaning of proactive AI at enterprise scale. The agent does not wander around the company looking for something to care about. The company tells the agent when something happened.

This distinction sounds small until it touches volume. Scheduled agents spend money on absence. Event-driven agents spend money on change. Once there are hundreds of agents watching thousands of queues, accounts, contracts, tickets, suppliers, documents, and policies, that difference stops being elegant architecture and becomes the operating cost of the AI program.

The agent runtime also needs a real lifecycle. Wake is only the first transition. A production agent should be able to acquire a work item, load its last checkpoint, take a lease on the relevant state, reason, call tools, write back a new checkpoint, and release the lease. If the work remains unresolved, it may stay active. If it is waiting on a human, a downstream system, a cooling-off period, or a future business event, it should hibernate with an audited state record.

That is the technical line I would draw. Perpetual loops are certainly not inherently wrong. Unbounded loops are. The problem is an agent that keeps thinking because nobody gave the runtime a safe way to stop, persist, and resume. A good design lets an agent stay awake while the decision is alive, and forces it to sleep once the next meaningful transition belongs somewhere else.

I think there are three somewhat unglamorous pieces that matter.

• The first is a trigger layer. Business systems have to publish state changes in a form other systems can trust. If a high-value account becomes a renewal risk, the signal should not live only as a human-readable note in a Teams channel. It should become a routable event.

• The second is an agent registry. Every serious agent needs a declared surface area: events it cares about, entities it owns, state it maintains, tools it may use, actions it can propose, actions it can take, and conditions under which it must hibernate. Without that registry, orchestration becomes vibes at machine speed.

• The third is audited state persistence. A hibernating agent has to remember where it left off, and the enterprise has to know what state it preserved. It should not rebuild its world from prompt fragments every time a scheduler wakes it. The account risk posture, last commitment, escalation history, open decision, pending dependency, and allowed next actions belong in durable state with a reason for the next wake-up.

This is where Enterprise Context Management becomes more than a governance phrase. Context is the condition that lets the agent resume intelligently after the business changes. It includes the event that woke it, the state it carried into hibernation, the business meaning attached to the customer or inventory item or policy, and the record of what the agent did next. At AI One, this is the architectural layer we think enterprises will have to get right as agents move from chat into operational workflows.

Governance must also be in the room, but I would not let it run the meeting. The more urgent implementation error is simpler: we are seeing teams use reasoning where infrastructure should do the waiting. That choice will show up as cost, latency, duplicate work, noisy escalations, and agents that seem busy while missing the moment they were supposed to catch.

It also changes vendor evaluation. Connector lists are table stakes. Scheduled runs are table stakes. Teams integration is table stakes. In a serious architecture review, I would ask for the wake trace: the source event, the routing decision, the agents left asleep, the checkpoint loaded into the agent that woke, the hibernation policy, the action boundary, and the record after execution. That trace tells you whether the product has an architecture for proactive AI or a scheduling feature with a model behind it.

OpenAI and Anthropic’s current workspace-agent direction is already pointing enterprises toward shared agents that can run workflows across tools, schedules, and applications like Email and Teams. That is a useful bridge because it gets companies thinking beyond chat sessions. The next step is richer than a schedule. Proactive AI needs a business event fabric underneath it and a runtime that knows when to stop thinking.

The cost pressure will help force this discussion. Axios reported Altman saying OpenAI’s top token user was using about 100 billion tokens per month, and that cost had become a huge issue amongst clients. Whether or not your enterprise is anywhere near that scale, the pattern is clear: once AI leaves the pilot budget, waste becomes visible. A single scheduled agent is easy to ignore. A fleet of agents polling every account, queue, channel, supplier, policy, and exception all day is a bill with an architecture diagram attached.

The right preparation is a wake-map exercise. Put one workflow on the wall. For renewal risk, write down the few events that deserve intelligence: severity-one ticket, usage drop, champion departure, procurement delay, legal blocker, failed implementation milestone. Draw the agents beside those events. Mark which agent owns account state, which one can act, which one should only recommend, what checkpoint it must write before sleeping, and which events wake a human immediately. If nobody in the room can say where the agent sleeps between those moments, the design is still a scheduled automation wearing an agent badge.

Altman also said that if there were one thing to prepare for over the next year, he would pick proactive AI. I agree. The preparation starts in the design review, with one awkward question on the table: what wakes this agent, what state does it carry back to sleep, and why is the model still thinking after the business event has moved somewhere else?

In the last two weeks, the three leading frontier labs have each reached for the same instrument. OpenAI launched the OpenAI Deployment Company with more than $4 billion in backing at a $10 billion valuation, anchored by TPG, Bain Capital, Brookfield, Goldman Sachs, McKinsey and Capgemini. Google Cloud CEO Thomas Kurian announced plans to hire hundreds of forward deployed engineers to form a new team inside Google Cloud. Anthropic partnered with Blackstone, Hellman & Friedman and Goldman Sachs on a $1.5 billion venture to embed engineers inside private equity portfolio companies and redesign workflows around Claude. The framing in each case is roughly the same: enterprise AI is hard, customers need help, send the engineers. This is a symptom of where we are in the adoption curve, not the answer to it. So what does the answer actually look like, and what are customers actually buying?

This rise of the forward deployed engineer tells us that the market has found a real problem: AI does not become valuable just because a model is available, a workflow has been demoed, or an integration exists in theory. The hard part begins when the system meets the customer’s actual operating environment, where the real process is not quite the documented process, handoffs are messy, ownership is unclear, approvals depend on context, and exceptions are often known by everyone but written down by no one.

In that environment, it makes sense that companies are sending engineers closer to the customer. Someone has to understand how the work actually happens before software can do anything useful with it. The mistake is to assume that because forward deployment is often necessary at the beginning, it is what the customer ultimately wants to buy.

The customer does not wake up wanting a forward deployed engineer. They do not want another technical team embedded in their business, another delivery model to manage, or another standing implementation function that quietly becomes part of the operating model. What they want is simpler and much harder: they want the work done, inside the environment where the work already happens, with enough reliability and control that they can trust the outcome.

The first use case is allowed to be messy

There is nothing wrong with using people to get the first use case right. In enterprise AI, it is often the only honest way to begin.

The first deployment is where the real workflow is discovered. It is where you learn which system is trusted, which process is mostly performative, which exception matters, where the risk sits, and what evidence the customer needs before they will let the software act with confidence. These things are not always available in a requirements document because the requirements document is often a partial fiction. It captures what the process is supposed to be, not necessarily how value, judgment, and accountability actually move through the organization.

That is the useful role of the FDE. They get close enough to the customer to expose the context that the product does not yet know how to capture for itself. They translate operational reality into something software can understand, and they find the gap between the demo and the deployment.

But that should be the point of the motion: to teach the software. If every new deployment requires the same amount of human discovery, bespoke configuration, and engineering effort to make the customer successful, then the company has not found a scalable AI model. It has found a consulting model with a better interface.

Consulting starts where product learning stops

The line between forward deployment and consulting is not the title on the team, it is whether the work compounds.

If an engineer sits with the customer, understands the workflow, builds around the exceptions, and leaves behind a system that makes the next use case faster, more reliable, and less dependent on human interpretation, that is product learning. If the same engineer, or another one just like them, has to come back and repeat the same exercise again and again, that is consulting.

This distinction matters because AI companies can easily hide a services business inside a software story. They can call it deployment velocity, customer obsession, outcome orientation, or any other phrase that makes the human effort feel strategic. Some of it is strategic. The danger is not being close to the customer; the danger is becoming permanently dependent on that closeness.

Customers will accept human help when it accelerates value, but they will not confuse it for the outcome. A long-running FDE engagement may feel reassuring at first because it means smart people are paying attention. Over time, though, it starts to look like exactly what customers were trying to avoid: another project, another dependency, another team whose knowledge lives partly in software and partly in people’s heads.

The promise of AI in the enterprise is not that customers get a more technical consulting team, it is that more of the work can move into software without losing the context, control, and judgment that made the human process work in the first place.

Forward deployed software

That is why the more interesting idea is forward deployed software.

Not software that waits for the customer to adapt to it, and not software that sits outside the workflow asking people to come to yet another place to get value. Forward deployed software enters the customer’s operating environment, learns how work is actually done, and gradually takes on more of the deployment burden that would otherwise sit with an FDE.

This does not mean pretending every workflow can be fully automated. That is usually where AI products become either brittle or irresponsible. In many enterprise processes, judgment still matters. Subject matter experts still matter. Review, escalation, auditability, and deterministic controls still matter. The point is not to remove humans from the loop everywhere; it is to stop using humans for work that software should be able to handle once the first deployment has taught it what matters.

Forward deployed software should learn the workflow, remember the exceptions, and understand where probabilistic reasoning is useful versus where deterministic logic is required. It should know when a task can be completed automatically, when it needs human review, and what evidence must be preserved so the customer can trust the result. In other words, software should start taking on the tasks that made forward deployment necessary in the first place.

That is what changes the economics for the customer. The first use case may require a heavier lift, but the second should be easier and the third should be faster. By the tenth, the customer should not feel like they are starting again. They should feel the return compounding.

The customer buys the outcome

Context graphs, integrations, and workflow memory all matter. But they are not what the customer is buying. They are the machinery behind the outcome. The customer buys the confidence that the work will happen correctly, consistently, and with less human effort over time.

That is the useful way to think about the infrastructure layer. It matters because enterprise AI cannot own real work if it does not understand the environment it is operating inside. It needs access to the systems, rules, exceptions, permissions, approvals, and prior decisions that shape how work actually gets done. But those things only matter if they make the outcome faster, safer, cheaper, or more repeatable.

No customer is buying a context graph because they want a graph. They are buying fewer stalled processes, faster approvals, cleaner handoffs, less operational drag, and more confidence that the same result can be delivered again without assembling another project team around it.

That is the customer-centric test for AI deployment. Did the work get done? Did it happen in the environment where the business already operates? Did it require less human effort over time? Did the next use case benefit from the last one?

If the answer is yes, then forward deployment has done its job. If the answer is no, then the FDE has become the product, and the customer is back in the world of consulting, even if everyone is using more modern language.

The real prize

The real prize is not more forward deployed engineers. It is software that can do more of what forward deployed engineers are currently being asked to do: understand the customer’s environment, translate messy workflows into executable systems, preserve the right controls, involve experts where judgment matters, and turn the first deployment into a faster path for the next one.

That is the difference between a services motion and a compounding software-based outcome. It is also the difference between a customer buying another project and a customer buying repeatable ROI.

Forward deployed engineers can be the bridge, but if you need them for too long, the bridge has become the destination. Customers do not want the bridge. They want to get to the other side.

The common thread isn’t really about pricing or packaging, it’s about rerunning the same playbook. These incumbents are taking the org chart we built for scarce human intelligence and laying it down as the template for abundant machine intelligence, and the framing is intuitive enough that most buyers aren’t questioning it.

Anthropomorphism was a useful bridge but it’s the wrong destination.

When ChatGPT launched, making AI feel human (conversational, personable, a little quirky) was a smart move because it lowered the barrier. People talked to it like a colleague and it worked. The anthropomorphism served a purpose: it made an alien technology approachable.

The problem is what happened next. A UX choice that worked at the chat interface got promoted into an operating model, and the metaphor that helped a single user feel comfortable talking to a chatbot is now being used to describe how entire enterprises should deploy, govern, and manage AI at scale. That’s a much bigger claim, and the metaphor doesn’t stretch that far. Making something feel familiar to a first-time user is not the same thing as designing how an organization should run.

Roleification places unnecessary limits.

“Paul the Paralegal” and “Sally the Salesperson” are agents defined by the specialization constraints of existing knowledge workers: an HR Agent that can only do what an HR person does, a Finance Agent bound by the walls of the finance department.

When you “onboard” an agent into a role, you are telling it to think within those walls. Of course, the agent does need to respect the access controls, data boundaries, and policies that define the task, and that part genuinely matters. But beyond those guardrails, the whole promise of generalized intelligence is that the thinking doesn’t have to be bound by the same walls that the job description is. Within what it’s permitted to see, the agent should be free to reason across whatever the work requires. Roleification optimizes for familiarity, making AI look like something a manager already understands, and familiarity is not the same as capability. The gap between the two is where all the value leaks out.

The work, not the worker.

The reframe that matters is this: stop thinking about what agents are and start thinking about what work needs to get done.

Take employee onboarding, which doesn’t really live in HR. It touches recruiting, IT provisioning, facilities, payroll, compliance, and the hiring manager’s calendar, and it’s slow today precisely because it’s distributed across six departments with their own systems, handoffs, and queue times. The HR Agent framing doesn’t fix this, it just automates HR’s slice and leaves the handoff friction untouched.

The more useful approach isn’t deciding which department an agent belongs to, but what outcome the business actually wants, what information is needed to get there, and what the fastest path to completion looks like. That path will almost always cut across the silos you’ve built, which is the whole point. The organizations starting to see real returns from AI aren’t the ones that bolted agents onto existing processes, they’re the ones willing to rethink the processes underneath.

This is process reengineering, not headcount planning.

Vendors don’t lead with this message, because process reengineering is hard, unsexy, and impossible to sell as a per-seat license, but it’s where the value actually sits.

Some of the incumbents are starting to acknowledge, at least in their marketing, that the game is moving. Salesforce going headless is the most visible example: the company that invented per-seat SaaS is repackaging itself as an API surface for agents, which is a recognition that the agent is the interface and the data layer is where the value will be captured, while also being a move to make sure they remain the ones capturing it. Our view has always been that the systems of record were going to have to become data companies, and exposing the API surface is the price of admission rather than a gift to the ecosystem.

When intelligence becomes abundant, and any node in your enterprise can reason, synthesize, and act, the binding constraint changes. It’s no longer whether you have enough smart people in the right roles, it’s whether your information architecture is set up to let intelligence flow to where the work actually is. That’s a fundamentally different question, and it demands a fundamentally different investment: not in agent personas, but in data flows, in breaking down the information silos that were built for a world where intelligence was scarce and expensive and had to be rationed into departments.

Your company’s competitive advantage was never the org chart anyway. It was the institutional knowledge, the customer relationships, and the proprietary processes, the things that actually differentiate you. Agents don’t protect that advantage by mimicking your current structure, they unlock it by making that knowledge actionable across every surface of the business.

The real infrastructure investment.

Getting this right starts with the layer underneath the agents. Before you deploy anything that calls itself an agent, you need a runtime that can govern what it sees, what it does, and what it’s allowed to conclude. Your proprietary data, business rules, and strategic priorities need to be extracted, structured, and made available as living context, rather than trapped inside department-specific tools or siloed in applications that don’t talk to each other. The question isn’t which agent you deploy, it’s whether any intelligence, human or machine, can access the right context, act on it safely, and be held to account for the outcome.

That means investing in how your organizational knowledge is modeled, how context is delivered to the point of work, how memory accumulates across interactions, and how every action an agent takes can be governed and audited. The cross-cutting layer where all of that lives cannot be owned by any single vendor whose data it’s governing access to. It has to sit on your side of the line, portable across models and across systems, or the independence you think you’re buying is an illusion.

When that foundation is in place, you don’t need to “hire” an HR agent or a Finance agent. You compose capabilities (reason, research, draft, decide, escalate) dynamically around whatever the work demands. You give your agents a goal and guardrails.

Where we think things will go.

The companies that will look back on this era and wonder what they were thinking are the ones building agent org charts today, meticulously onboarding digital employees into the same departmental silos that have been slowing them down for decades. Right behind them will be the ones who accepted a vendor’s headless pivot as the whole answer, and only discovered later that the runtime governing their agents was never really theirs.

The ones that will win are doing something less photogenic but far more consequential: rewiring how their business thinks. Not adding AI to the machine but rebuilding the machine around what AI makes possible.

Your agents don’t need job titles. Your enterprise needs intelligent plumbing.

That is why I increasingly think LLM agnosticism is becoming a strategic requirement for enterprise AI, not a technical preference or a purity argument. The recent Information Report on Anthropic’s changes to enterprise pricing for heavy business usage is significant because it shows how quickly the economics can shift once workloads become serious. Providers will price for their own compute constraints, capacity limits, and margin objectives, and they will work hard to pull customers deeper into their own hosted agent ecosystems. That is entirely rational from their side. It is simply not a stable basis on which to build the rest of your company’s operating mode in a market this volatile, this politically exposed, and this fast-moving.

The durable asset is not the model endpoint. It is the agentic system that governs how models interact with your data, memory, context, tools, and policy. If that layer belongs to you, providers compete for your workloads, the best models can be chosen task by task, and switching remains an engineering exercise rather than a strategic crisis. If that layer belongs to them, your economics, privacy posture, and exit costs are all downstream of somebody else’s platform strategy. Avoiding that trap requires more than a gateway. It will require owning the runtime.

Why This Is Becoming Urgent

The Anthropic pricing change is a useful signal because it exposes a deeper truth about agentic AI economics. Once usage shifts from seat-based productivity into long-running coding agents, scheduled work, tool use, retries, and autonomous workflows, pricing stops being a clean per-user abstraction and starts to reflect raw consumption. The Information piece suggests that some heavy Claude Enterprise users could see costs double or even triple under the new structure, and it notes similar usage-sensitive moves elsewhere in enterprise AI. This is probably not an outlier, but what happens when providers discover where the expensive workloads are.

And price is only one axis of dependency. The frontier model market is moving too quickly, its economics remain too unsettled, and its policy environment is too uncertain for any serious company to commit its architecture to one provider’s roadmap. You do not know which firms will still be structurally advantaged in three years, whose gross margins will hold, which models will end up constrained by regional regulation or procurement rules, or where governments will decide strategic control points need to sit. Frontier models are already close enough to strategic infrastructure that export controls, access restrictions, or safety interventions can no longer be treated as remote possibilities.

That is too much uncertainty to absorb passively. The practical response is to make model choice reversible.

The Best Model Depends on the Job

This matters because different model families genuinely excel at different kinds of work, and the best answer is rarely “pick one and standardize everything around it.”

In many enterprise teams, the picture has already shifted noticeably over the last three months. For coding workflows in particular, many teams have begun moving from Claude toward Codex, not because Claude has stopped being useful, but because OpenAI is often better at following precise instructions, operating inside tighter agentic control loops, and using tools in a more disciplined way over longer execution chains. Claude still has a distinct strength of its own: it is often better at finding a user’s underlying intent inside messy, ambiguous conversation, which matters in workflows where the hardest problem is understanding what was actually meant before any action is taken.

Neither choice should dictate the architecture of the whole company.

There is another advantage to model plurality that is still underused: one model can check another’s work. A model that proposes an action is often a poor judge of its own blind spots, especially if the same family is also being asked to validate the output, grade the result, and decide whether to continue. Once you let a second model from a different provider review the work, or better still attack it, you reduce correlated failure. One model writes the code change, another looks for the edge case; one proposes a customer action, another searches for the compliance or policy problem; one drafts the plan, another tries to break it. What matters is not merely a second opinion, but a second failure distribution.

That diversity also gives you a more rational cost structure. Not every step justifies the most expensive model, and not every task should be solved with the model you happened to start with six months earlier. The ability to route by task, validate across families, and change those choices without a replatform is what low switching cost actually looks like in practice.

The Real Lock-In Is Where Enterprise Meaning Lives

The strongest form of lock-in is not the API call. It is where your memories, context, ontology, and workflow semantics end up living.

Hosted agent frameworks are attractive because they compress months of work into days. They ship with managed memory, built-in tracing, default planners, integrated evals, native tools, and smooth developer ergonomics. That convenience is real, but so is the functional limit. Most are optimized for their own abstractions, their own storage assumptions, their own tool semantics, and their own view of governance. They tend to treat context as a provider-native payload rather than a governed architectural layer, which is exactly why they feel so productive at the start and so constraining later.

This is where sovereignty stops sounding abstract and becomes operational. Data sovereignty in this sense does not just mean “keep the files private.” It means the enterprise owns its memories, context layers, business ontology, approved tools, versioned queries, access rules, and audit trail. It means a model can invoke a reviewed and approved query by name rather than inventing live query logic in production. It means credentials remain at the gateway or data-access layer rather than inside the agent loop. It means the map of enterprise meaning, how customers, products, policies, mandates, exceptions, and actions are actually defined, stays on your side of the boundary.

That is the layer that compounds with use, and it is the layer you do not want to hand away.

We have seen this pattern before in the cloud. Single-provider choices did not usually hurt in year one; they became painful in year three, when data gravity, egress, proprietary services, and rewrite costs converted earlier speed into later dependency. AI providers are now trying, quite sensibly, to move up the same stack. They do not want to provide only inference; they want the hosted agent framework, the memory layer, the eval system, the tracing surface, and the developer workflow that makes your application harder to move. The easiest architecture to start with is becoming, again, the hardest one to leave.

Agnostic Does Not Mean Generic

There is, however, one important caveat that gets missed in simplistic “multi-model” discussions: an LLM-agnostic system is not just a gateway.

The transport layer is the easy part. The hard part is behavior. Different model families want different prompt structures, tool descriptions, context packing strategies, result validators, retry policies, stop conditions, and cache behavior. Real agnosticism does not mean pretending Claude, OpenAI, and a local open model are interchangeable. It means keeping the invariant layers, your data fabric, memory, ontology, policy controls, audit, and tool execution, on your side, while versioning prompt packs and task variants by model family and even by model version so each one is optimized for the work it is doing.

If you want low switching cost without dropping into lowest-common-denominator behavior, you need a canonical internal representation of prompts and tasks, with provider-specific translation at invocation time, and you need different prompt versions retained for different families because the same task often performs differently across them. In other words, low switching cost is not achieved by denying model differences; it is achieved by containing them.

This is one of the reasons we designed our ContextOne architecture in the way we did. The runtime is LLM-agnostic, but it does not assume identical behavior across model families. It preserves the enterprise’s control over memory, ontology, audit, and policy while allowing prompts and tasks to be optimized for specific providers and versions. That is the appropriate kind of complexity, because it buys flexibility where it matters and specialization where it pays.

Local Open Models Belong Beside the Frontier Loop

Once you own orchestration, smaller local open models stop being ideological and start being practical.

They can run in side-chains alongside the main agentic loop, handling tasks that are narrow, repetitive, privacy-sensitive, or simply not worth frontier-model pricing. A local model can classify inbound documents, normalize entity names across messy systems, rerank retrieved passages, or perform first-pass policy and code checks before the frontier model is called.

These side chains can run in parallel with the main loop, which means the gain is not only lower cost but also lower latency and reduced exposure. The frontier model spends its budget on reasoning rather than housekeeping, while the most sensitive or repetitive steps remain inside the enterprise boundary. For regulated businesses, that hybrid pattern is usually much better than the false choice between “everything hosted” and “everything local.”

Board-Level Takeaway

The strategic question is not which model vendor looks strongest this quarter. It is whether the company owns the runtime that governs data, memory, ontology, tools, and policy, because that layer determines pricing leverage, privacy posture, resilience, and exit cost. It is the layer that compounds with use. A company that owns it can let model vendors compete inside its system; a company that does not is effectively underwriting the platform risk of the vendor it chose first.

Conclusion

For most serious enterprises, the disadvantages of running their own agentic system in-house are front-loaded and manageable, while the benefits compound over time. You take on more architecture, more governance work, and more operational discipline up front; in return, you get lower structural risk, better cost control, stronger privacy, clearer ownership, reduced lock-in, and the freedom to use the best model for each task rather than the model you happened to standardize on early.

None of this means provider ecosystems are useless. They will remain valuable learning surfaces, and for many teams they will still be the fastest way to get started. But as soon as AI touches core workflows, proprietary data, and real budgets, convenience stops being the right optimization target. Risk, cost, lock-in, ownership, and privacy move to the front of the queue.

That is why the more durable pattern is to keep the context and control architecture on the enterprise side of the boundary, let frontier and local models work together inside it, and make model choice reversible. We build around that principle: LLM-agnostic at the runtime layer, optimized by model family where it matters, and structured so the enterprise keeps the memories, ontology, and controls that actually become more valuable with use.

The companies that benefit most from this cycle will not be the ones that guessed the eventual winning model provider. They will be the ones that refused to bet the rest of their architecture on that guess.

This is where we see some of the current conversations about “evals” breaking down. The term has become a catch-all for runtime guardrails, policy review, task verification, benchmark scoring, and sometimes deterministic controls that are not really evaluations in the first place. The lack of precision matters. If you collapse all of those into one category, you end up using the wrong instrument for the wrong job. The industry still tends to treat agent failure as a model-quality problem, however in our experience it is increasingly a runtime-design problem.

At AI One, we have been working on this problem inside ContextOne, our enterprise agent runtime. What has become clear is that evaluation only becomes reliable when it is treated as part of the architecture. Not a score at the end. Not a judge prompt bolted onto a swollen context window. A governed system that decides what the agent may do next, what it must prove before it continues, and what the platform should learn from the result.

These are early findings, not final answers. But they suggest that evaluation is less a single technique than a stack of distinct mechanisms, each matched to a different class of failure.

Why “Evals” Is Too Broad a Word

In traditional software, verification is easier because the system is deterministic. A function returns the right output or it does not. Agentic systems behave differently. The same model can take different paths to the same objective. It can look reasonable while violating sequence, policy, or downstream state. Once an agent can read enterprise data, call tools, and write to live systems, “does the response look plausible?” stops being a serious standard.

Manufacturing figured this out a long time ago. You do not use end-of-line inspection to decide whether a bolt should have been tightened earlier in the assembly process, and you do not use a torque sensor to decide whether the finished engine actually runs. Different failure modes require different checks at different points in the process. Agentic systems are no different.

Consider an investment-bank operations agent running a pre-open allocation workflow. If it allocates a block trade before checking inventory, that is a sequencing failure. If it proposes an allocation that fits the mechanics but breaches client mandate or desk risk posture, that is a judgment failure. If it reports the booking as complete but the order management system still shows the old state, that is an outcome failure. The stakes are not academic: broken trades, manual repair, capital and compliance exposure, and the familiar question of why was a proof of concept allowed anywhere near production?

In practice, four different questions hide inside the word “evaluation”:

1. Can the agent take the next step?

2. Is the proposed action aligned with policy and intent?

3. Did the claimed outcome actually occur in the underlying system?

4. Does the architecture materially improve autonomous task completion across realistic workflows?

A resilient system answers each with a different mechanism.

If those questions are collapsed into one category, system design gets confused. Teams ask an LLM judge to do work that deterministic logic should have handled. They use benchmark scores as a proxy for runtime safety. Every control gets called an eval and sight is lost of what each component is there to solve.

Runtime Evaluation Has Three Jobs

Our research indicated we needed to settle on a three tier architecture. Instant Eval asks whether the next move is allowed. The LLM Judge asks whether the move is aligned with higher-level intent. The Agentic Grader asks whether the job was actually done.

Instant Eval: Can the agent take the next step?

Instant Eval enforces deterministic, real-time constraints on sequencing and preconditions of tool calls. Before an agent can execute a tool, the runtime checks whether the required prior steps have occurred. If the check fails, the agent receives actionable feedback rather than a silent rejection.

In an airline workflow, if the agent attempts to reserve a seat before it has checked availability, the call should be blocked and the agent should be told exactly which prerequisite is missing. That sounds simple. It is also the difference between a self-correcting system and a corrupted record in production. A surprising share of enterprise failure is structural, not conceptual.

LLM Judge: Is the decision consistent with policy and intent?

Some decisions cannot be reduced to sequence. They depend on judgment, tradeoffs, and principles that are too broad or too fluid to encode as a strict set of preconditions. This is where the LLM Judge belongs. It sits at designated checkpoints and evaluates whether a proposed action aligns with higher-level objectives and constraints, using only the slice of context relevant to that assessment.

Consider a loan approval workflow. An agent proposes a $50,000 loan to a customer with a credit score of 620. The sequence may be correct. The decision can still violate the firm’s risk posture. That is a judgment problem, not a sequencing problem. This is where an LLM Judge is useful, and also where it should stop. It should not be asked to do work that deterministic logic can do better.

Agentic Grader: Did the work actually happen?

The final runtime question appears when an agent says it is done. In many deployments, this is still the weakest link. An agent can produce a convincing summary of a task it did not actually complete. It can claim success after making only part of the required change. It can update the wrong record and still sound confident.

The Agentic Grader verifies the outcome against the underlying system. For tasks that can be checked deterministically, it can call Python scripts, database queries, or gateway tools directly. For tasks that require a more interpretive check, it can dispatch dedicated sub-agents with introspection tools and feed their output into a separate judge. If an agent says it updated a customer’s seating assignment and the reservations system does not show the change, the task is not complete. The agent goes back and fixes the discrepancy.

That separation matters because agents fail before an action, at the point of action, and after the work is supposedly complete.

Some Controls Should Not Be Left to Evals

One of the most common confusions in agentic AI is treating every control surface as an eval. Some mechanisms exist to judge behavior or outcome. Others exist to make certain classes of failure impossible. Both matter. They do different jobs.

Formal constraint enforcement is the clearest example. Some business rules are too important to leave to probabilistic judgment. In ContextOne, those rules can be translated into logical constraints and checked with a formal constraint solver before execution. If a proposed action violates the rule, the action is blocked. A loan approval that conflicts with a formal credit policy should not be “graded down” later. It should fail closed.

In regulated environments, query generation belongs in the same category. For example, in ContextOne, we use the concept of Named Queries to let the agent invoke fixed, versioned query structures by name, supplying parameters rather than generating live query logic at runtime. Credentials stay out of the Agent Harness, access is enforced at runtime, and activity is recorded in a cryptographically signed audit ledger. These are not alternative flavors of eval. They are the architectural conditions under which evaluation becomes meaningful.

Why This Belongs in the Runtime

A useful evaluation framework cannot sit outside the agent as a wrapper around one giant prompt. It has to live where the work is being done. In ContextOne, that is the Agent Harness.

The architectural split is simple but important. The loop reasons and the harness governs. That separation creates a stable place to apply runtime checks, manage permissions, validate tool calls, gate results back into context, and halt or escalate when thresholds are reached. In ContextOne’s governed loop, we made that explicit: prompt assembly, context management, model invocation, result validation, tool execution, result collection and context gating, and control condition evaluation all happen inside a managed execution pipeline. Large tool outputs are offloaded and referenced rather than blindly stuffed back into context. Cost and iteration limits are checked before an agent is allowed to continue.

The economics matter too. ContextOne is stateless and event-driven. Agents materialize when work is required, yield resources between iterations, and resume from shared state in single-digit milliseconds. That makes governed, step-by-step execution practical at enterprise concurrency.

The practical takeaway is simple: a model that looks intelligent in a proof of concept is not production-ready until you can explain what stops it, what judges it, what verifies its work, and what it is deterministically prevented from doing.

Benchmark Evaluation Answers a Different Question

Runtime evaluation tells you whether a live task should proceed. Benchmark evaluation answers a different question: does the architecture materially improve autonomous task completion across realistic workflows?

That is the role benchmark evaluations play. On Tau Bench 2, ContextOne achieved 95% task completion in the telecoms domain against a baseline of roughly 30%. In the airline domain, it achieved 78% on a manually verified solvable subset against a roughly 40% baseline, and 52% on the full task set against a roughly 25% baseline. The same prompts were used for both the baseline and ContextOne runs. The gains came from architecture rather than prompt optimization.

But benchmark evaluation should not be confused with runtime safety. A Tau score does not tell you whether a particular loan approval, trade allocation, or customer update should be blocked right now. It tells you whether the architecture has moved the system from constant-supervision territory toward genuine delegation. At around 30% task completion, an agent still demands babysitting. At 90%+, you are starting to talk about real business process execution, with human oversight reserved for genuine exceptions.

The Board-Level Test

Before any agent moves from proof of concept to production, management should be able to answer four plain-language questions: what blocks it, what judges it, what verifies that the work actually happened, and what it is deterministically prevented from doing. “The model looked good in testing” is not a control framework.

What Comes Next

What comes next is not a search for one universal eval. It is better architecture for separating questions that are currently collapsed into that one word.

Which actions should be blocked deterministically? Which decisions should be reviewed against principles? Which outcomes must be verified against live system state? Which failures belong in a benchmark, and which belong in the runtime? Which signals should feed memory and improve the next execution? These are the design questions that matter.

And this connects directly to memory. In our earlier SubStack article on memory, we argued that enterprise agents need governed ways to retain what happened, how tools should be used, and which contextual facts matter. Evaluation is what makes those memories trustworthy. Memory without evaluation compounds both skill and bad habits. Evaluation without memory catches the same failure forever. Joined together, they create a feedback loop in which successful trajectories become reusable patterns, failed actions become correctable guidance, and autonomy can expand only when the system has earned it.

Our current findings point in one direction. In enterprise AI, evaluation architecture may be one of the most under-leveraged variables in system performance. Models matter, and prompts matter. But in agentic workflows that touch real systems, the more consequential lever may be the architecture that determines what an agent is allowed to do, what it must prove, what it is prevented from doing altogether, and what the platform learns from the result.

We do not claim to have the final answer. The evidence so far suggests that the systems that earn production trust will be the ones that know when to block, when to judge, when to verify, and when to learn. They will treat evaluation not as a single score, but as a governed part of the runtime itself. That is the standard proof-of-concept projects should be held to, before anyone calls them production-ready.

At AI One, we’ve been tackling this “memory problem” for large enterprises. We’ve created a Memory Engine with three asynchronous pipelines that create a feedback loop between agent execution and agent learning. Each pipeline captures a different class of knowledge from each run and injects it into the next. These are early findings, not final answers. But they suggest structured memory may be a more consequential lever than model sophistication or prompt engineering.

This article shares our approach.

Why Memory Is the Frontier

Every serious engineering system has a feedback loop. Gas pipelines have PID controllers (Proportional, Integral, and Derivative). In machine learning, gradient descent makes training observable, debuggable, and repeatable. Software teams have CI/CD. The dominant agentic frameworks have none. Every invocation is independent. Context is assembled fresh each time: conversational history, tool definitions, and retrieved documents are appended into a single payload that grows until it hits the token limit. Nothing carries forward.

The consequences are many. An agent queries a technology catalog where the same vendor is listed as “IBM” in one system and “International Business Machines Corp.” in another. The agent fails on entity resolution. It failed on the same resolution yesterday. It will fail again tomorrow. An agent calls an API that returns 15,000 tokens, consuming budget that should go to reasoning. Another agent violates a business rule. Nothing stops it from repeating the same mistakes over and over.

The LLM isn’t the problem. The problem is the absence of a feedback loop to retain important learnings, discard what’s irrelevant, or compound skills over time.

Three Pipelines, Not One Memory

Our approach consists of three asynchronous pipelines. Each handles a different class of learned knowledge: episodic, procedural, and semantic. Their design is guided by the idea that agents need to remember different kinds of things; each requires distinct storage, retrieval, and update mechanisms. All three run in parallel with the main agent loop.

Episodic Memory: Trajectory Distillation

The first pipeline captures episodic memory: what happened.

Every agent execution produces a trace: reasoning steps, tool calls, and outcomes. Episodic memory pipelines review these traces, cluster similar activities, and identify “golden trajectories” — execution paths that represent best practices for a given task class.

This isn’t log storage. The distillation process compresses full execution histories into reusable patterns. When an agent encounters a similar task later, it starts from the best-known path, not from zero. It retains the freedom to deviate when specifics demand it. Agents get faster and more accurate on repeated task classes. No manual prompt tuning required.

Procedural Memory: Tool Learning

Enterprise agents interact with dozens or hundreds of tools: APIs, databases, and internal services. Each has its own quirks, failure modes, and optimal invocation patterns. The procedural memory pipeline captures how to use a tool by analyzing tool outcomes and updates prompts that guide future tool use.

For instance, imagine an agent that queries a technology catalog where the same vendor appears under three names across three systems. First attempt: entity resolution fails. The procedural pipeline captures that failure, analyzes the tool’s response, and updates the guidance. Next invocation: the agent accounts for the naming inconsistency. We update our procedural memory again. And on it goes. Over time, the agent’s skill with each tool compounds.

The system doesn’t just remember that a tool exists. It remembers how to use it well.

Semantic Memory: Contextual Facts

The semantic memory pipeline captures facts. It extracts contextual information from agent interactions: business rules, user preferences, organizational hierarchies, and domain terminology. It builds a persistent, governed model of the enterprise’s operational reality.

Semantic memory is what allows an agent to know, without being told each time, that “net revenue” means something different in the EMEA division than in North America. Or that a particular approval workflow requires two VP sign-offs, not one. These facts accumulate. Because they’re stored in a structured layer rather than buried in conversational history, they can be shared across agents, audited, and corrected.

Context Management Memory Strategies

The three pipelines feed a catalog of context management strategies. These strategies are invoked during each agent iteration. They include techniques such as token-efficient reformatting, result offloading, interaction summarization, noise injection, and prompt cache management. Let’s examine two of these strategies in detail to illustrate problems that only surface in production and that no amount of prompt engineering can fix.

Noise Injection: Breaking Degenerate Loops

After lengthy sequences of repetitive tool calls, LLMs can become trapped in degenerate loops. The model produces the same sequence of actions, receives the same results, and repeats. Each individual step is technically correct. Deterministic controls can’t catch it because no single action violates a rule. The failure is emergent: the model has been, in effect, lulled by the regularity of its own output.

This failure mode was first documented by the Manus team. We address this by injecting noise: small, semantically meaningless perturbations introduced into the context flow at controlled intervals. These perturbations disrupt the repetitive pattern without altering the task’s semantics. The agent breaks out of the loop and resumes productive reasoning. It’s a pragmatic, empirically validated countermeasure to a problem that exists at the boundary between deterministic system design and probabilistic model behavior.

Prompt Cache Management: Co-Designing Context and Cost

Context management systems reformat, offload, summarize, and mutate context between iterations. Each mutation can invalidate cached prompt prefixes. This matters because prompt caching is one of the most effective cost and latency optimizations offered by modern LLM providers. Naively applying context management without considering cache implications results in a perverse outcome: reduced token counts per call but higher effective costs due to constant cache invalidation.

One way to address this problem is a “harness-managed prompt caching” strategy that’s aware of the full context management pipeline configuration. The harness knows which regions of the context are stable (system prompts, ontology definitions) and which are volatile (tool results, summarized history). It structures the prompt to maximize prefix reuse even as downstream content changes. This optimization is only possible when context management and cache management are co-designed rather than treated as independent concerns. The platform provides dedicated views into context stability and cache performance, so operators can tune strategies not just for accuracy but for economic efficiency.

The broad design choice of context management strategies is that they happen at the architectural level, not the prompt level. Each content class (tool results, conversational history, ontological facts, memory-derived guidance) is optimized independently. This keeps context windows small and clean as task complexity grows, instead of the monotonically ballooning payloads that characterize monolithic approaches.

Memory and Human-in-the-Loop Governance

Reducing business rule violations doesn’t come from memory alone. It requires deterministic controls, including an SMT solver that enforces formal logical constraints on agent behavior. Memory is what makes those controls practical.

Without memory, every invocation is a cold start. The agent has no record of which actions trigger compliance issues, which tool sequences produce reliable results, or which edge cases require human escalation. A memory-free workflow catches violations reactively; with memory, the agent is less likely to violate a rule in the first place. The result? Fewer interruptions. Fewer escalations. A more efficient human-in-the-loop workflow.

This is “tunable autonomy.” Enterprises need to adjust the tradeoff between speed and accuracy, cost and confidence. Human in and out of the loop. Memory helps make those tradeoffs. An agent with rich episodic and procedural memory can operate with higher autonomy because it has demonstrated competence. An agent encountering a new domain should operate with tighter controls. Memory is the mechanism that distinguishes the two.

The Compounding Effect

The most important property of the Memory Engine, based on what we’ve observed so far, is that its knowledge compounds continuously. Each execution improves the next. Skills learned from one agent’s interactions become available to others through the shared ontology. Tool guidance refined through procedural memory applies across every agent that uses that tool. Contextual facts persist and accumulate.

This separates a learning system from a stateless one. A stateless agent’s performance is bounded by prompt quality and context window capacity. A learning agent’s performance is bounded by the breadth of its accumulated experience. Over time, the gap widens. The stateless agent stays flat. The learning agent improves.

Early manufacturers who adopted electricity didn’t see transformative gains until they redesigned their factories around the technology, replacing centralized steam engines with distributed electric workstations placed where the work happened. The same principle applies to LLMs and memory. Bolting an LLM onto existing workflows produces marginal improvements. Redesigning the architecture so agents accumulate knowledge, refine skills, and operate within governed memory structures produces a different class of system. One that gets better with use.

What Comes Next

What comes next is not a search for one universal memory architecture. It is a better way to decide what should be remembered, what should be forgotten, and what should be treated as canonical. Episodic, procedural, and semantic memory have different update patterns, retrieval requirements, and failure modes, so any serious enterprise memory architecture has to remain extensible as new domains and new operational realities appear.

The harder problem is deciding which memories matter the most for the task in hand. Some memories carry more weight because they are recent. Others should dominate because they encode stable business rules or durable tool-use patterns. Some because they were decisive in a prior successful trajectory, even if they were not the most obvious part of the trace. That attribution problem is where our focus is now: ranking memories, using temporal lineage, and ascertaining which prior experiences materially improve the next execution.

Those answers will not come from benchmarks alone. They will come from production deployments, under real constraints of governance, latency, and cost, where compounding effects become measurable over weeks and months: skill transfer across workflows, fewer repeated failures, lower human intervention, and improved token economics.

Our early findings point in one direction. In enterprise AI, memory architecture may be one of the most under-leveraged variables in system performance. Models matter. Prompts matter. But in repeated, governed workflows, the more consequential lever may be the memory architecture that determines what an agent keeps, reuses, and learns from. We do not claim to have the final answer. The evidence so far suggests that the next major gains will come from agents that do not merely reason at inference time, but actively accumulate governed experience over time.

Production-grade observability for AI agents requires a different architectural principle: one that elevates data lineage to a first-class observable, captures the complete causal chain from data source to agent decision, and maintains this chain across agent boundaries, tool invocations, and temporal gaps.

First, let’s explore why observability in the context of AI agents is different.

Why AI Agents are Harder to Observe

The operational monitoring of software systems has matured considerably, converging on structured logging, OpenTelemetry-based distributed tracing, and Prometheus-style metrics. For traditional distributed applications, this stack provides a recoverable causal chain: a trace shows which services were called, logs show what each service did, and metrics reveal the resource conditions.

AI agents break this model. An agent processes a request according to its code, its prompt, the current contents of its context window, the results of any tools it has called, and the probabilistic reasoning of an LLM. Two identical requests, submitted seconds apart, can produce entirely different outcomes—not because of a system malfunction, but because the data retrieved by a tool call changed, the context window was managed differently due to token pressure, or the LLM’s sampling produced a different reasoning path.

This non-determinism is challenging enough for a single agent. It becomes dramatically harder to reason about in multi-agent systems, where agents consume one another’s outputs as inputs. In these environments, a log entry stating “Agent B processed the data” tells an operator almost nothing about why the output contains what it does, where the underlying data came from, or whether it was transformed in transit. The existing observability stack can tell you what happened. It often fails to explain why the agent decided what it decided, and where the data that informed that decision came from. For enterprise deployments, where audit, compliance, and explainability are non-negotiable, this is a critical gap.

The Three Dimensions of Agent Observability

Meaningful observability for AI agents must operate across three dimensions that traditional tooling addresses unevenly at best.

1. Operational Observability: What Happened

This is the dimension best understood by existing tools. Distributed traces show the sequence of service calls. Metrics capture latency, throughput, and error rates. Logs record discrete events. The industry has a robust ecosystem, largely built around the OpenTelemetry standard, that provides this foundational layer of visibility. This dimension answers the first, most basic question an operator asks, but for AI agents, it is rarely the last.

2. Decisional Observability: Why It Decided What It Decided

The second dimension requires capturing the full reasoning context that led to an agent’s decisions. This is more than just a log of the final output; it’s a snapshot of the agent’s complete execution context: the system prompt, the tool results present in the context window, the conversational history, and the specific content of the LLM’s response.

A new class of LLM-native observability tools, such as LangSmith and the open-source Langfuse, has emerged to address this. They capture detailed traces or trajectories of an agent’s execution, providing a step-by-step view of its internal operations, including the full content of the context window at each decision point. This is a significant and necessary step beyond traditional APM. The OpenTelemetry project’s GenAI semantic conventions are formalizing a standard vocabulary for these traces, defining attributes for gen_ai.input.messages, gen_ai.system_instructions, and tool call results. However, treating decisional observability as a separate, specialized form of tracing risks isolates the why from the where. A trace that shows an agent made a decision based on a piece of data is useful; it becomes operationally essential when that trace is natively linked to the full, verified lineage of that data.

This requires more than capturing additional data in a trace; it requires structuring that trace as a queryable record of the agent’s full decisional state, where every element in the context window is not just a string, but a reference to a verifiable source.

3. Provenance Observability: Where the Data Came From

The third dimension is data lineage. When an agent acts on a piece of information, the enterprise must be able to trace that information back to its source: the specific query, table, or API call that produced it. When that information flows between agents, the lineage must be maintained without interruption.

Open standards like OpenLineage provide a crucial foundation, offering a vendor-neutral framework for capturing lineage metadata from data pipeline components at the point of access. This is essential for dataset-level lineage. However, agent observability requires a finer granularity. It’s not enough to know that Agent A used a dataset that came from a specific database. We need to know that a specific cell in a spreadsheet generated by Agent B is a result of a transformation of a specific record retrieved by Agent A at a precise moment in time. The W3C PROV standard and its recent extension, PROV-AGENT, demonstrate that the academic community has been developing the conceptual models for this level of granularity. The production challenge is implementing these models at the architectural layer of an agent runtime, not as an instrumentation library.

This requires lineage to be more than a record reconstructed after the fact; it must be an intrinsic, inseparable property of the data itself. Lineage should be stamped at the point of access and propagated automatically through every transformation and every agent handoff. Critically, access controls must propagate with lineage. If data carries a sensitivity attribute—“US Staff Only,” “MNPI”—that attribute must not be lost when the data is transformed or repackaged by a downstream agent. This is not something that can be easily bolted on; it must be enforced by the underlying architecture, making data sensitivity a structural property of the system.

Multi-Agent Observability: Maintaining Lineage Across Boundaries

The challenges of these three dimensions compound in multi-agent architectures. When a single agent fails, a combination of operational and decisional observability might be sufficient. When multiple agents collaborate, a new class of failure emerges: failures of interaction that are invisible at the individual agent level.

Consider a scenario where Agent A retrieves customer data, Agent B applies a risk model, and Agent C generates a recommendation. If the recommendation is incorrect, the root cause could lie in any of the three agents, in the data itself (which may have changed between retrieval and consumption), or in a subtle reasoning error by Agent C operating on correct inputs. Without a continuous, cross-agent lineage chain, diagnosing this requires a manual, error-prone reconstruction of the data flow.

True multi-agent observability requires a single, connected graph of provenance that allows an operator to trace any output back to its original sources through every intermediate transformation. This is not a correlation of independent logs from different agents; it is a unified view of the complete data flow across all participating agents, their tool calls, and the external systems they accessed.

Cryptographic Integrity and the Future of Audit

For regulated enterprises, observability data is not merely operational—it is evidence. The integrity of this evidence is paramount. While not yet a mainstream practice, the use of cryptographic techniques to ensure the immutability of audit trails is a logical and necessary evolution. W3C Decentralized Identifiers (DIDs) provide a path to giving each agent instance a verifiable, persistent identity, enabling cryptographic signing of traces, trajectories, and lineage records. A hash-chain-backed log, as demonstrated in the AuditableLLM framework, can ensure that the observability record is tamper-evident and independently verifiable—a requirement that will become increasingly common in finance, healthcare, and defense. The key property is that verification must be possible without access to the platform that generated the records; the cryptographic proof must be self-contained.

Conclusion: From Patchwork to Platform

The observability tools the software industry has built over the past decade are necessary but insufficient for the unique challenges of AI agents in production. They answer what happened, but struggle with why the agent made the decision it did and where the data came from. As enterprises move from single-agent experiments to multi-agent production deployments, this gap will become the primary obstacle to operational confidence, regulatory compliance, and organizational trust.

The path forward is a lineage-first approach to agent observability. This does not mean abandoning the principles of logs, metrics, and traces. It means re-centering them around a new architectural principle: a unified graph of data provenance. The principles are straightforward: capture lineage at the point of access, propagate it automatically through every transformation, enforce access controls as an inseparable property of the data, and provide cryptographic guarantees that the record is trustworthy.

The challenge is not conceptual; it is architectural. The solution lies not in better tools applied after the fact, but in an architecture that makes observability an inherent, structural property of the system. This means treating the lineage graph as the primary data structure of the agent runtime, not as a secondary artifact produced by instrumentation. Logs, traces, and metrics remain essential; they become views into the graph rather than independent streams. Access policies become edge attributes on the graph rather than gateway rules. Cryptographic signatures become properties of graph nodes rather than optional add-ons. When observability is structural, it cannot be omitted, misconfigured, or bypassed. That is the architecture the enterprise AI industry needs to build toward.

I don’t mean we hid it behind a feature flag or stopped talking about it. We removed it from the product, ripped out the code paths, and stopped designing around it. It wasn’t a rage decision. It was the end of a slow, unmistakable realization: the builder was training us to solve the wrong problem.

When we first shipped it, it felt like progress. Engineers could point to a canvas and say, “This is what happens.” Boxes, arrows, a clean sequence from input to output. It made the system easy to grasp, especially for people seeing it for the first time. But easy to grasp isn’t the same as correct.

Not all “agents” are the same

At some point we had to admit we were mixing up different categories of “agentic” products. From a distance, it can look like everything is an “agent” now. But under the hood, it’s really a few distinct paradigms wearing the same label.

The first is the classic visual workflow builder. But it’s not really an agent. It’s a workflow: a structured, predefined sequence where the user specifies the path up front - usually with drag-and-drop blocks. It’s great when the operation is repeatable and the world stays stable. But it doesn’t reason. It doesn’t adapt. It mostly just advances.

The second is what I’d call the framework-first approach - the ecosystems where you wire up tool calls, loops, and multi-step behaviors with prompts and orchestration code. Those tools made “agent development” accessible. But they also push the hard problems into the least reliable place: the model’s behavior. The more you rely on prompt compliance for governance, the more your security and policy guarantees become aspirations instead of structure.

The third is the power-tool category: things like coding agents and lightweight task runners. They can be shockingly capable in a constrained environment. But they usually don’t give you the things organizations need when the work matters: centralized policy enforcement, reproducibility, consistent audit logs, and a way to explain why it made the decisions it did.

We realized what we were building is closer to a fourth category: an enterprise-aware agent system. The experience should feel as immediate as the best power-tools - type a goal, give it tools, watch it work - but with the operational guarantees that make it deployable in the real world: enforceable boundaries, evidence and lineage, and auditability.

The First Shift: From Steps to Outcomes

Back to our visual builder. The first cracks didn’t show up in the demo environment. They showed up in the unglamorous places: the runs where the data didn’t match the happy path, where a tool returned something plausible but incomplete, where a downstream system changed a response shape without telling anyone. The builder did what it was built to do: it moved forward. It advanced to the next box.

And that’s exactly when we started losing the plot.

Because the system we were trying to build wasn’t supposed to “advance.” It was supposed to reach a goal reliably, even when the path wasn’t the one we expected.

That was the first shift: we stopped thinking in steps and started thinking in outcomes. In a builder, the path is the artifact. In an agentic system, the path is disposable, a means to an end.

Once you see that difference, you can’t unsee it.

We tried to reconcile the two worlds for a while. We added more structure. More branches. Retries. Validation blocks. Error handlers. We made the canvas smarter the way everyone makes canvases smarter: more boxes. The diagrams got impressive—dense, “enterprise,” serious.

They also got fragile.

Every new block was a commitment to predict the future: which failure modes matter, which recoveries are safe, which validations are sufficient, which conditions are worth branching on. And as soon as production taught us a new lesson, we encoded it as another node. That created a familiar pattern: the canvas wasn’t representing reality; it was chasing it.

Here’s the distinction we couldn’t escape: workflow graphs are execution plans. You pre-author a path. Agentic systems are control policies—they observe, decide, act, and verify against the goal. Adding boxes just hard-codes yesterday’s lessons; it doesn’t buy you adaptation.

The Second Shift: From Calls to Actors

The second shift made the decision inevitable: we stopped treating the model like a single call and started treating it like an actor in a system.

The best runs shared a simple shape:

• give the model a goal

• give it tools and skills

• constrain it with guardrails

• let it cook

The surprise was how often the right path differed. The model changed tool order based on what it found, recovered from failures, asked clarifying questions when needed, and validated intermediates because the goal demanded it.

A concrete example made this obvious.

Take onboarding. Not “fill in a form” onboarding, real onboarding where inputs arrive as emails, PDFs, spreadsheets, and half-structured notes. One run is clean; the next has mismatched identifiers, missing evidence, or a policy threshold that changes what checks are required. The system has to ask a question, route to a reviewer, pull a source of truth, re-check assumptions, and then prove the outcome is supported.

In a canvas world, that’s not one workflow. It becomes a library of workflows: branches for every policy fork, variants for every product line, region, customer type, evidence format, and exception path. Over time it turns into hundreds of slight variations, most of them identical except for one step, one validation, one integration edge case.

An agentic system treats those variations differently. The goal is stable. The constraints are explicit. The tools are bounded. The agent decides the path at runtime based on what it sees, and it can explain why it took the route it did.

That’s what “agentic” meant in practice. It is not a buzzword, but a behavior: understanding the environment, adapting to change and error, and staying oriented to the goal.

And that’s where the canvas became more than inconvenient. It became actively counterproductive.

A visual workflow builder rewards certainty. It rewards pre-definition. It encourages you to treat the journey as sacred: draw it carefully, lock it in, and the system will follow it faithfully.

But effective automation isn’t faithfulness to a plan. It’s faithfulness to an outcome.

The builder was making the journey more important than the destination.

We felt this most sharply in debugging. With a canvas, a run “fails” when a node errors. But many of the worst failures in automation don’t throw errors, they produce outputs that look reasonable. A workflow graph can happily deliver something that passes through every box and still be wrong, because the world changed and the flow didn’t notice. The diagram stays clean. The result quietly degrades.

To solve that, you don’t add more arrows. You add a control plane: explicit checks, verifiable tool boundaries, consistency tests, and guardrails that stop the system from “hallucinating to completion.” You make the system able to say: I can’t support this conclusion with evidence. Or: I need clarification. Or: This output fails validation; I’m taking a different approach.

Those are not canvas nodes. They’re governance primitives.

The Third Shift: Operations Change Everything

The third shift was operational: supporting deployments over time means living in continuous change. Data formats drift. Vendors update. Policies and thresholds move. Exceptions emerge. Tools develop new failure modes. Teams change how approvals and escalations work.

Once you account for that, the math is brutal: every diagram is an artifact you now have to keep in sync with a moving world. It starts as “a few workflows,” becomes a library, then a forest of near-duplicates.

Onboarding makes the cost obvious. If step X exists in 500 variants, a small upstream change isn’t one fix, it’s 500 edits, 500 tests, and 500 chances to miss a corner.

In an agentic system, the unit of change is smaller and more powerful. You improve a tool. You refine a prompt. You add memory. You tighten a guardrail. One change, applied everywhere. The system adapts at runtime instead of demanding that engineers pre-adapt it in diagrams.

The Decision to Delete

Once we accepted that, keeping a workflow builder around wasn’t harmless. It was pulling the product toward a worldview we no longer believed: that the primary job of automation is to predefine the path.

So we deleted it.

Not as a rejection of visual builders in general, just an acknowledgment of what we were actually building. In an agentic-native system, the interface isn’t a canvas. The interface is principles, intent, constraints, tools, and a trace of decisions that you can inspect, audit, and replay.

The goal stays fixed. The route is allowed to change because the world does.

And after we committed to that, a workflow builder wasn’t a feature anymore. It was a contradiction.

So how do you actually translate this shift into shipped systems?

Start With the Right People

Not every developer will thrive in this new model. Some have always coded for the beauty of code itself rather than for the goal of producing working business systems. AI agents will displease those who love coding for its own sake, and empower those who love to ship real outcomes.

Find a handful of those people and experiment with them first. I don’t believe there’s a strong level of experience correlation here; what matters is mindset. Specifically, look for developers who:

• Are curious and willing to learn. Working with AI agents is a new discipline. The people who will succeed are those who approach it with genuine curiosity and a willingness to experiment.

• Are driven to ship business value, not just code. The agent handles much of the raw coding. What remains is the harder, more critical work of translating business problems into working systems.

• Understand the business itself. People who grasp what the organization is trying to accomplish will be far more effective at steering agents toward the right outcome. This matters more than ever, because the bottleneck is no longer typing speed, it’s judgment.

• Have used AI coding tools and can talk about what they’ve done. There’s simply no excuse for not having used a number of tools at this point. If you meet someone who hasn’t committed to that growth mindset and at least done personal work with it, start to ask a lot of questions.

Pick the Right Problems

Once you have one or more small teams, the challenge is finding the right kind of problem to experiment with. AI agents work best on greenfield projects, though they absolutely help with legacy systems too.

Regarding scope, the obvious answer is often the right one - you should start small with a single end-to-end feature. However, more importantly, you must start with something verifiable. AI agents shine when they can validate their own work. Focus on problems with clear, measurable results - a well-defined API, a calculation that can be checked, a workflow with observable outputs. Avoid anything ambiguous where success is subjective and hard to confirm.

Recognize Where the Pain Shifts

This is a point that catches many organizations off guard. The development itself is no longer the long pole. The pain moves upstream and downstream: to specification and to review.

Before an agent can build the right thing, someone needs to clearly articulate what the right thing is. This loops back to having developers with a solid handle on the business problem being solved. They should also have a strong understanding of architecture, organizational standards, and deployment constraints so that the agent doesn’t produce something technically impressive that can never actually ship.

On the other end, review becomes the critical bottleneck. AI agents can generate substantial volumes of working code, but someone still needs to verify that it does the right thing, in the right way, within the right guardrails. Plan for this. Build review capacity into your workflow from the start - and that doesn’t have to be manual.

To make this a bit more concrete: inside the product engineering team at AI One, we have shifted design reviews to the left: the story initiator writes the initial design document (AI-assisted), one or more other team members review it and provide feedback, and then we assign a single developer to take it forward from that point to production. During development, we use AI tooling (alongside heavily automated, high coverage end-to-end testing) to perform reviews across a number of different dimensions: correctness, functionality, and security. Then, once the developer has proven the story is in a state of readiness, they submit the PR for another team member to review. The team member then reviews the evidence and the high risk areas of the PR before approving it (or requesting changes). This sits around level 3 to level 4 of Dan Shapiro’s Five Levels of AI coding. As a SOC II compliant organization, we require at least some human oversight of the code so this approach fits well into a highly regulated context.

Invest in Infrastructure and Test Environments

This follows naturally from the review constraint. AI agents perform dramatically better when they can interact with real systems in controlled environments. If you’re building something that connects to a custom internal API, give the agent access to a live instance of that API in a sandbox. If this cannot be achieved, build as close a digital twin as you can. Then, let it validate that API calls actually did what it expected.

Agents are remarkably good at feeling their way around systems and figuring out how best to use them once given real access. Static documentation alone is a poor substitute.

A Practical Example: Building Integration Gateways

We recently needed to build gateways for both Google Workspace (authentication and access to Drive, Sheets, Calendars, and more) and Microsoft 365 (authentication, Teams, SharePoint, and related services). Rather than handing agents documentation and hoping for the best, we stood up full sandbox environments for them to work against.

We populated these environments with real data - files in SharePoint and Drive, channels in Teams, calendar entries - and gave the agents well-defined tasks: authenticate, retrieve a file from SharePoint, list items in a Drive folder, post to a Teams channel. The agents had full access to these sandbox environments, access to the internet for API documentation, and crucially, the ability to verify every step of their work against real system responses.

The result was high-quality integration gateways that were proven from the start to work correctly. They weren’t built in theory and then tested, they were built through testing, with the agent iterating against live systems until everything behaved as expected. We still reviewed the output for security, architectural fit, and edge cases, but the heavy lifting - the tedious, iterative work of getting OAuth flows, API pagination, error handling, and data mapping right - was all done by the agents.

This is the pattern that works: give agents real environments, clear objectives, and the ability to validate their own results. The quality of the output is dramatically higher than what you get from agents working blind against documentation alone.

Then, critically, ensure the agent builds out the tests necessary to enable quick and safe changes in the future. The first build is just the beginning - the test suite is what makes it sustainable.

Don’t Overthink Costs - Invest to Experiment

It’s surprisingly uncommon for developers to be given truly high-quality tools - whether that’s powerful laptops, top-tier IDEs, or $200/month subscriptions to AI agents. Use your small teams to experiment and build the business case - it’s a very small investment for a potentially massive return.

As a closing piece of advice, I’d also strongly encourage organizations not to be too restrictive about letting curious developers use their subscriptions at home for personal projects. Building a small side project, automating something in their own life, experimenting with a new framework - this is all part of learning to work effectively with the tools. That fluency will flow directly back into their professional output. Treat it as professional development, because that’s exactly what it is.

The number itself is less important than the precedent. The implicit demand is clear: if AI made you faster, I want the discount.

That demand exposes a long-standing economic tension. The buyer’s instinct is understandable, but it rests on an assumption that has always been fragile: that hours billed are equivalent to value delivered. The billable hour has historically functioned as a proxy for value in professional services, not the value itself. But in reality, clients do not purchase time; they purchase judgment, accumulated context, risk transfer, and accountable outcomes. When AI compresses the hours required to produce an audit, a diligence report, or a strategic memo, it does not automatically compress the expertise or liability embedded in the final deliverable.

What the KPMG example illustrates is not just fee pressure in auditing. It signals that “AI pass-through” is becoming a baseline expectation. Wherever a buyer can point to a visible process and ask, “How many hours is this still taking, and why?”, pricing models anchored to labor time will come under scrutiny.

The mechanism will vary by industry, but the pressure is similar. In software and professional services, AI reduces labor minutes. In operational settings—factories, logistics networks, supply chains—it reduces variance: fewer defects, fewer returns, less downtime, tighter inventory. In both cases, measurable efficiency gains create procurement leverage.

Service providers will respond by arguing that AI does not merely make the same output cheaper; it shifts the quality frontier. A faster audit that surfaces anomalies with greater precision, a compliance workflow with stronger traceability, or a supply chain with materially lower variance is not strictly the same product at a lower cost. It is a product with different performance characteristics. From the vendor’s perspective, the relevant question is not “How many hours did this take?” but “How much risk was removed, how much upside was unlocked, and how much better is the outcome?”

The problem is that procurement teams rarely price on abstract improvements in quality when visible cost savings exist. The adjustment is unlikely to be linear. The adoption path more closely resembles a J-curve followed by an S-curve.

The J-curve is the awkward phase in which efficiency gains exist but are difficult to monetize cleanly: duplicated workflows, model risk controls, data integration overhead, security reviews, and persistent edge cases that pull manual work back into the loop. During this period, prices may not fall meaningfully, and in some cases may even rise as vendors absorb transition costs.

The S-curve follows once tooling standardizes, model workflows are audited, and AI is embedded in operating procedures. At that point, procurement gains confidence in the stability of the new process and begins pricing against unit economics rather than prototypes. Explicit AI discounting becomes harder to resist. Once a few vendors concede, peers risk appearing overpriced overnight.

This creates acute pressure for firms that lag in implementation. Vendors unable to demonstrate credible efficiency gains—or unable to absorb pass-through pricing—will struggle to compete.

Yet the long-run dynamic is more complex than simple deflation. When technology meaningfully increases the productive capacity of expertise, demand often expands. As AI raises the ceiling on what a professional can analyze, oversee, or optimize, the binding constraint shifts from labor time to judgment and accountability. Pricing anchored purely to hours becomes increasingly incoherent, but total value creation may grow.

Whether buyers pay for that expanded capability or simply capture the efficiency gains depends largely on market structure. In highly competitive markets, savings are more likely to pass through, producing pockets of disinflation even as quality improves. In concentrated markets, vendors may retain a larger share of the productivity dividend, expanding margins while performance rises.

For firms early in their AI implementation cycle, the practical constraint is not model performance but proof. Efficiency gains that cannot be measured, governed, and allocated are difficult to defend in pricing negotiations. Buyers demanding discounts will increasingly demand evidence: auditable trails that separate real throughput improvements from shifted risk or hidden rework.

The evidentiary standard cannot stop at hours saved. If the debate is framed purely in terms of cycle time or headcount compression, the buyer’s logic dominates. The more durable strategy is to make surplus legible across multiple dimensions: throughput, error reduction, variance compression, regulatory defensibility, risk transfer, and outcome quality.

When a buyer can quantify not only cycle time and exception rates, but also control strength, auditability, and decision quality with the same rigor applied to financial metrics, the conversation changes. It shifts from “How much labor was removed?” to “What new level of assurance or performance is now possible?”

The audit fee reduction is therefore less about one negotiation and more about a structural question that will echo across sectors: when AI increases productive capacity, who captures the surplus? The answer will depend on competitive intensity, evidentiary discipline, and how effectively vendors re-anchor pricing away from time and toward accountable outcomes.

This is why the concept of “context graphs” entered the conversation, and why they matter. The recent framing is precise. Context graphs are not knowledge graphs. They are decision trace graphs. They preserve what mattered at decision time so that justification, precedent, and accountability do not dissolve into chat logs.

That is a meaningful advance. It is crucial to include historical decisions, exceptions, and edge cases when making decisions about how to act. But over the last year, when actually implementing enterprise wide production systems, we quickly found that it is not the end state.

The mistake many teams make is subtle. They correctly observe that decision traces are consulted during agent reasoning, and then quietly slide from “informs action” to “authorizes action.” Once that line is crossed, the architecture starts to accumulate risk in ways that only appear months later.

What follows is the perspective that emerged when we treated decision traces as advisory reasoning surfaces, and deliberately placed authority elsewhere.

Why The Context Graph Concept Invites Overreach

In practice, the strongest solutions in this area are active participants in agent reasoning. They are queryable during planning and execution in order to surface precedent, exceptions, constraints, and prior approvals. Used this way, decision traces materially improve reasoning quality. They ground agents in organizational memory, reduce hallucination, and make decisions legible after the fact. This is precisely what makes the graph concept so tempting to treat as a source of authority. Graph nodes look executable. They encode conditions, actions, and rationale. It is easy to slide from “this informs what we should do” to “this permits us to do it.”

That slide is where risk accumulates. A true context graph would preserve what mattered at decision time, but would not enforce what is valid now. When allowed to authorize action, three failure modes reliably appear. Precedent outlives policy because time is represented but not enforced. Rare exceptions become reusable templates as agents optimize for citation rather than validation. As graphs grow richer, systems become better at justifying actions than at checking whether they are still allowed. These are not flaws in graph design. They are the natural outcome of treating memory as control.

Separately, in our practical experience, we found that trajectories solve a different and equally important problem. A trajectory is the time ordered trace of how work actually unfolds, including goals, actions, failures, escalations, overrides, and outcomes. Reasoning over trajectories shifts agents away from inventing plans and toward reproducing successful ones. Recovery improves because failure patterns are visible. Planning becomes empirical rather than speculative. But trajectories are descriptive, not normative. They explain how work gets done here, not what is permitted to run.

Behavior Before Permission

The missing primitive is executable authority. Rather than storing justification as narrative memory, we embedded it into time versioned units of execution - skills. A skill is not a suggestion. It is an enforcement point. Each skill defines the action it permits, the preconditions that must hold, the authoritative sources it may consult, and the subsequent deterministic validations that must be passed, including business rules, policy constraints, and risk controls. It also specifies any approvals or exceptions required, the evidence that must be emitted, and the window in which the skill is valid. Skills are evaluated synchronously at execution time and fail closed. If the rules no longer pass, the policy no longer applies, or the justification has expired, the action does not run. This is the moment where “why” stops being an explanation and becomes control.

Time bounding is the property that turns governance from an aspiration into a system behavior. Without it, organizations slowly accumulate what we came to think of as precedent rot. Temporary workarounds harden into defaults. Policy interpretations linger long after their intent has expired. Nothing breaks loudly, but authority quietly drifts. By contrast, a time bound skill expires in the open. It must be refreshed from sources, revalidated against policy, or deliberately replaced. Drift becomes visible and actionable instead of silent. Decision traces remain invaluable in this process because they preserve the history of how and why decisions were made. But they can only observe drift. Skills are what stop it.

When combined deliberately, the division of labour becomes clear. Different primitives serve different roles, and problems that once felt entangled separate cleanly. Trajectories ground agents in how work actually succeeds by exposing real sequences of action and recovery. Decision traces ground reasoning by preserving precedent, rationale, and explanation. Skills sit at execution time and decide what is allowed to run now, under current conditions. Agents consult all three continuously, but only one has the authority to say yes. This resolves a tension many teams feel but rarely articulate. Decision traces (or implementations of context graphs) should absolutely influence what an agent proposes to do. They simply should not be the thing that permits it.

From Reasoning to Control

The enterprise choice is straightforward. Either reasoning is allowed to imply permission, or permission is enforced independently of reasoning. Only one of these scales safely. Systems built on trajectories and skills are much easier to transition from prototype to production, and far harder to break once they are there.

But the more interesting question is not whether forward-deployed work is dangerous. It’s when it becomes dangerous, and what actually separates a compounding platform from a high-end delivery shop in an era where software itself is getting cheaper and faster to build.

In practice, many of the most valuable enterprise AI problems still demand real proximity to the customer. The hardest work lives in messy, high-stakes domains where production outcomes matter: reconciling exceptions across fragmented systems, continuously monitoring disparate systems for early risk and next-best-action signals, and capturing the “in-between” context - decisions, SOPs, undocumented overrides - that determines whether automation is explainable, auditable, and trusted. These are not problems that vanish with better models alone.

The mistake is assuming that because this work is necessary, it should be permanent.

Forward deployment should be a use-case scaling strategy, not an operating model. The first phase - going from zero to one - benefits enormously from tightly integrated field and platform engineering. But that phase only creates leverage if it is explicitly designed to end. The purpose is not to deliver forever; it is to extract a repeatable pattern that can be scaled with far less human involvement.

This is where the article’s warning is most important, and where execution discipline matters more than rhetoric. Time-boxing initial deployments, constraining customization to the edges, and protecting a stable, upgradeable core are not nice-to-haves. They are the mechanisms that force compounding. Without them, every new customer quietly fragments the product, and the organization accumulates entropy rather than advantage.

Where the piece feels underweighted is on how moats actually form in the 2026 timeframe.

The traditional platform moat, large inventories of bespoke integrations and proprietary primitives, made sense when software was expensive and slow to build. That world is fading. AI-assisted engineering is collapsing development time and cost, which means static platforms are easier to replicate and harder to defend. The connector layer, by itself, is no longer durable.

What compounds now is not the surface area of the platform, but the operational playbook behind it. The ability to repeatedly drive real 0→1 outcomes, transition those outcomes to 1→N scale, and feed the resulting learnings back into the system. Most critically, it is the accumulation of context and memory: entity maps, workflows, decisions, state, and institutional knowledge that make each deployment smarter than the last.

That “context layer” becomes a durable advantage for customers, much like data did in prior eras, but only if the system remains modular and upgradeable. Lock-in through rigidity is brittle; advantage through adaptability is not.

Finally, the article implicitly treats SaaS as the default endpoint business model. That assumption is increasingly questionable (and probably requires its own post based on the stock market events of this week!). As reliability improves and margins converge toward software economics, outcomes-based pricing becomes viable - first in hybrid form (platform plus outcomes), and increasingly outcomes-weighted over time.

The key distinction is that outcomes cannot simply be layered on top of bespoke delivery. The winners will be companies that can tie outcomes to platforms that genuinely compound, where each success lowers the cost and increases the reliability of the next one.

The real risk in enterprise AI isn’t forward deployment. It’s mistaking early momentum for durable leverage, and failing to design for compounding from day one.

A piece lives in your CRM’s new chatbot. Another is in the BI tool that turns questions into SQL. A third is in the agent framework your engineering team uses. These fragments don’t talk to one another. They can’t reason across your business. It’s a flawed foundation that won’t scale.

There’s a better way: a single, coherent system of context. A brain you own and control. When you own the context you feed AI, you control how your AI reasons about every question your business asks. This requires a better architecture.

We’ve previously argued that the Hybrid Loop is the gold standard for building AI systems that think. This essay is about how to train that brain. The strategic question isn’t how your AI reasons, but what it reasons with.

Two Paths to AI Failure

Most leaders see two paths for building with AI. Both are traps that cause you to lose control of your enterprise context.

Path 1: Build it yourself on a legacy framework. Developers designed toolkits like LangChain and CrewAI when models were weak and could only reason one step at a time. Their solution was to chain models together, letting them improvise in a loop. But this creates a contextual mess. Each step adds noise and replays history inefficiently. There is no central source of truth. The framework designed to help you instead creates the chaos you must overcome.

Path 2: Rent it from a SaaS vendor. This path lets your vendors handle it. Your CRM gets a copilot. Your document store gets a summarizer. This approach is seductive and offers quick wins, but the long-term cost is control. Your customer data, product information, and internal processes are now managed inside someone else’s black box. You cannot standardize it. You cannot move it. You have outsourced your company’s memory. The result is a lobotomized organization. Your sales AI can’t learn from a support ticket, and your support AI can’t see a customer’s purchase history. Every interaction starts from zero.

Chaos or a lobotomy. Neither is a good choice

There’s a better way.

The Third Path: Own Your Enterprise Context

There is a third road: treat your enterprise’s context as your most valuable asset, managed by a dedicated system. This is the discipline of Enterprise Context Management, or ECM. ECM is like a library and head librarian for your AI. It doesn’t just store the books; it knows which page to open for any given question. The library is organized, the librarian is precise, the building is secure, and you choose who gets a library card.

When you own your context, you own your company’s brain. The lessons you’ve learned and inferred principles live in your library, not in a vendor’s application or scattered in code. It enables the AI Hybrid Loop architecture to function at scale based on intelligence that’s organized in multiple layers (like a Context Skyscraper).

What does this brain look like in practice? It has five key functions:

• A Centralized and Curated Context Catalog. Instead of disconnected data or metadata silos, ECM is built on a Context Catalog. As explained in the introduction to Enterprise Context Management, this catalog doesn’t just store data or create semantic data views. It enriches context with operational data. It knows which customers are new, which contracts are expiring, and which internal processes are critical. It catalogs the history of the prompts that worked and didn’t. It enforces privacy. Your AI uses this context to reason, ensuring every decision is based on a complete picture of reality.

• Principle-Based, Not Rule-Based Context. Our human brains are pattern-matching engines. We see a new situation and instantly process it based on previous experience. AI makes judgements in much the same way, based on predictions and inference. Business context, therefore, must be designed for fuzzy, principle-based judgements and predictions. So ECM is built on principles that guide decisions based on context.

• Context Delivered with Precision. A catalog is only useful if the right information can be accessed at the right time. An ECM platform shapes and delivers the exact context needed for each step of a task. For a high-level planning step AI takes, it leans on strategic principles. For a detailed verification step, it might consider a single, critical data point. This precision allows the AI to operate on flexible principles derived from your business.

• A Secure Control Plane in Your Environment. Owning your brain means keeping it secure. ECM platforms run within your own environment, close to your data, respecting your data residency and privacy rules. It is managed by your team, not a third party. This is enforced through a control plane that implements a zero-trust security model. Every request from a user or an AI agent is authenticated and authorized, operating on a principle of least privilege. This ensures that as AI gains autonomy, it does so within the boundaries you defined.

• Control Over Your AI Models. An ECM platform separates the brain (your context) from the voice (the AI model). This gives you the freedom to choose, mix, and swap the foundational AI models that interact with your context. You are not locked into a single vendor’s ecosystem. When a better, faster, or cheaper model arrives, you can adopt it without having to rebuild your entire enterprise brain. You own the asset; the model is the tool you choose to apply to it.

The Context Brain in Action

Your enterprise now has a brain, a central library of its own. But a library is only as good as the wisdom you can draw from it. The next step is to teach it to think. Forget what you know about rules. The core insight is this: agents need principles plus context, not data plus rules.

The old era of rules was about looking backward. You mined historical data to create rigid checklists. You told your system to halt a process if any box was unticked. The logic was brittle because it lacked situational awareness. It knew the rule, but it didn’t understand the customer or the goal.

The ECM era gives agents that awareness. It lets them apply your business principles, even when the data is ambiguous. That ability to navigate subjectivity is a feature, not a bug. It is judgment.

Imagine AI managing new customer onboarding. The old rule is simple: halt the process if any required document is missing. A high-value enterprise customer is ready to go, but a non-critical setup form is missing. The system stops. The customer waits. Momentum is lost. But AI with a context brain operates on a principle: prioritize onboarding steps based on customer value and compliance risk. It sees the missing form but also knows the customer is strategic and the document is not a legal requirement. It allows the technical integration to proceed while flagging the missing form for the account manager to handle personally. It distinguishes between an administrative snag and a genuine roadblock. This is not just automation. This is intelligence.

A principle-driven brain doesn’t just execute tasks; it understands priorities. It learns to apply the same nuanced thinking your best people use.

This is how your business already operates, through unwritten principles:

• In banking, you prioritize high-value clients, not just high-value transactions.

• In pharma, you escalate patient reports of unusual side effects, even if they are not on an official list.

• In your supply chain, you review uncommon shipping delays, not just any delay over 48 hours.

• In insurance, you assess claims where multiple risk signals converge, not a dollar amount.

• In marketing, you review AI-generated copy that sound robotic.

• In HR, you investigate atypical patterns in employee expense reports, not by dollar amount.

• In manufacturing, you flag unexplained dips in production quality, not every deviation from the norm.

• In your legal team, you prioritize review of contracts with non-standard liability clauses.

• And in customer service, you escalate support tickets with language that indicates frustration.

AI can make these principle-based judgements. But each requires context and a process to apply it in the right way at the right moment. AI that runs on principles, powered by the unified context of your enterprise brain, can finally provide it. It can act not just on what the data says, but on what the situation demands.

The Strategic Choice

The choice before you is not about which AI tool to use. It is about what you want your AI to be and how your initial pilots can work at scale. Do you want AI to come from a third party? Serendipity? Or do you want to control your brain and harness the wisdom of your data, people, and processes?

AI from a third party can tell you who’s in your CRM. AI with wisdom can tell you which customers are active on Zendesk and determine they may be at risk of leaving and how to save the relationship. That leap from data to wisdom is impossible when your company’s memory is fragmented or outsourced.

ECM platforms are the engine of that transformation. They gather scattered context and turn it into knowledge. But once you stop feeding agents with data and rules and start feeding them with principles and context, you unlock enormous upside and expose all the messy, unsolved problems of principle design, debugging, and accountability. That’s the “next chapter” of enterprise AI that AI One has been treating as a personal mission.

Once agents are operating on principles instead of narrow rules, their effective agency increases. That creates a new class of enterprise challenges: how you represent principles in policy, how you audit decisions that were made under uncertainty, and how you explain “why the agent did that” to regulators and risk teams. This is the governance surface we’ve been obsessing over for the past two years, and it will be the topic of our next post.

This is how you create an AI that truly grows with your business. The Hybrid Loop may be the blueprint for an architecture that thinks, but the brain itself is made of context. When you own that context, every question your employees ask makes the AI smarter. Every decision becomes part of its permanent memory. Every relationship it maps deepens its understanding. This is not rented intelligence. This is the enduring mind of your enterprise.

This essay describes the architecture that resolves this conflict: the hybrid loop. It is written for leaders who know LLMs are powerful, but who now must build durable, real-world systems that think. The core idea is simple:

• Let the model think deeply and globally at a few key points.

• Let deterministic code own execution, state, and safety.

• Let a hybrid agentic architecture guide the whole thing.

Before we examine the hybrid loop, consider the architectures that came before it.

From Stepwise Agents to Single Shot Planning

In the first wave of LLM systems, the model was a short-term problem solver. It could not hold a long chain of reasoning. It had limited context. Its planning was weak and its tool use was fragile. So engineers built what they had to: stepwise agents.

The pattern was always the same. The model would:

1. Look at the current context.

2. Propose a small next action.

3. Trigger a tool or fetch some data.

4. See the result.

5. Decide what to do next.

Each decision was another call to the model. For anything complex, you quickly had long chains of calls. Every step replayed most of the past context. Every step added a little more noise. And every step was a chance for the model to drift. Frameworks emerged to manage these chains: strict REACT style, Planner-executors, Multi agent, self-refinement, and a raft of legacy orchestration frameworks such as LangGraph, CrewAI and AutoGen to construct and support them. But they all shared the same fundamental flaws:

• Latency and cost grew with the number of steps.

• Long chains were brittle and hard to debug.

• The same queries behaved differently when the context order changed.

• There was no clean way to define success or set budgets.

Then the models changed. Context windows grew. Tool use became more reliable. Planning improved. The best models started to feel less like chatbots and more like strong planners. They could take a clear goal and lay out a multi-phase approach in one shot.

This created a new temptation: Single Shot prompting: push everything into one big prompt, ask for the full plan and the final answer, and be done with it.

Sometimes that works. If the environment is static and the tools are simple, a single planning and execution pass can be enough. But most business problems are not like that. Markets move. Data changes. APIs fail. The Single Shot method is too rigid for the real world.

The chain of agents is too brittle. The Single Shot is too blind. A third way was needed.

The Hybrid Loop: Planning, Supervision, and Verification

A better way has emerged: the Hybrid Loop. It starts from a different division of labor. Instead of asking one model to “be the agent” and do everything, it separates the task into clear roles and underlying manager. This combines the strengths of today’s powerful models with the control of iterative execution.

The hybrid loop is an architecture of five actors, each with a distinct role. Four of them perform the work; a fifth, the Context Manager, directs the flow of information between them.

First, there is a planner model. This is a frontier reasoning LLM used for what it is now very good at: taking a high-level goal, a list of tools, requirements, and constraints, and producing a structured plan. Not a paragraph of text, but a multi-phase object that describes what to do, in what order, and with what success criteria.

Second, there is a supervisor. This is not a model. It is regular code. It holds the current plan and the execution state. It owns tool calls, asynchronous fanout, parallelism, budgets, and state transitions. It knows nothing about language. It just executes and enforces.

Third, there is a verifier. This combines deterministic code and LLMs, but has a narrow mandate: judge, never act. It is fed compact context snippets, like a generated SQL query, a JSON structure, an LLM output, or even execution traces. It produces1 structured labels such as “valid,” “block,” or “unsafe2.” These labels then feed back into the Supervisor.

Fourth, there is the tool layer: your APIs, databases, and other services. That is where the actual work happens. Tool calls are asynchronous, often parallel, and frequently the dominant cost and latency in the system. They are made by a combination of code, and LLM instantiated calls.

Underpinning them all is the Context Manager, the operation’s central nervous system. It sends targeted pulses of information to each actor. Just what they need, exactly when they need it. Nothing more, nothing less.

Once you see the architecture in these terms, the question is no longer “how do I chain agent steps,” but “how do I build clean interactions between these roles.”

The loop is “agentic” because the system as a whole acts like an agent: it turns goals into actions. It is “hybrid” because the intelligence is shared between models and code in a deliberate way.

A Day in the Life of a Hybrid Loop

To make this concrete, imagine a standard business task: a user asks for a set of risk reports. This requires joining data across multiple systems, selecting the right tables, generating valid SQL queries, and writing a report.

Here is what happens inside a hybrid loop.

The planner model sees the request, a description of available tools, and a set of constraints. Instead of improvising step by step, it returns a plan object. That plan might say:

• Phase 1: Discover relevant tables using a catalog search tool.

• Phase 2: Fetch detailed schema information only for candidate tables.

• Phase 3: Generate one or more SQL queries that answer the question.

• Phase 4: Execute the queries and check the results.

• Phase 5: Write a report and send it to the user.

This plan is not created in a vacuum. The Planner constructs it using a curated set of information provided by the Context Manager, which draws from a Context Catalog of available tools, data sources, and constraints, organized in levels (for more on the architecture of context, read The AI Context Skyscraper on this Substack)

The resulting plan includes success criteria (e.g., “at least N candidate tables with a date column and a measure column,” hard constraints (“never use tables whose name starts with archive_ or tmp_”), budgets, (maximum tool calls or time per phase), and instructions on how to summarize tool outputs before they’re sent back to a model.

The supervisor picks up this plan and begins execution. During phase 1, it might call a table listing tool ten times, often in parallel, with different search terms. It collects and filters the results according to selection rules in the plan, like ignoring certain tables or requiring a time dimension. All of this is done in code, not by a model.

To generate the query in Phase 3, the supervisor does not send the model a raw data dump. It asks the Context Manager for a surgical strike. The Manager prepares a tailored package of information of the user’s goal and a lean summary of candidate schemas. The model can then reason without distraction.

If the stakes are high, the supervisor can call a verifier model with the SQL and schema summary. The verifier does not try to rewrite anything. It simply labels the query: for example “valid, warn” or “invalid, block, attempts to access unauthorized PII data3.”

If the label is acceptable, the supervisor proceeds. If not, it records the problem and decides whether to ask for a new plan or to stop.

All the while, the supervisor is monitoring whether the plan’s expectations are being met. If the plan is not working, it can prepare a summary of what has happened and send that back to the planner for a patch or a complete replan.

The hybrid loop is not a free-form conversation. It is a state machine that moves through planning, execution, verification, and replanning, with models and tools playing defined roles at each stage.

When the Plan is Wrong

In a toy example, the first plan might be perfect. In the real world, plans can be wrong. The hybrid architecture assumes this and makes being wrong a first-class concern.

When the plan does not match the world, there are two basic responses.

If the plan is broadly sound but a detail is off, the supervisor can ask the planner for a patch. It sends a small evidence pack, and the planner responds with changes. The supervisor applies those changes and continues.

If the plan’s assumptions are fundamentally broken, the supervisor can ask for a rewrite. This time it compiles a richer summary of the environment. The planner uses that to construct a new plan. The supervisor switches to this new plan and starts again.

There is a final, crucial guardrail for when the system itself loses confidence. The hybrid loop can be designed to know what it doesn’t know. If the planner is stuck, or if a result falls outside a defined boundary of certainty, the system stops and escalates. It can invoke the most reliable tool of all: a human in the loop.

In both cases, the combination of planner, supervisor, and verifier lets the system change course without becoming opaque. Plans, patches, and rewrites are all explicit objects that can be logged, compared, and audited.

Context is a Design Surface

Raw context is noise. The hybrid loop works because it treats context as a design surface, and the Context Manager is the designer. It derives intent, builds the complete context window, and optimizes each section using the most efficient techniques for that section.

The context manager keeps the world view the planner needs that’s rich enough to propose a sensible strategy, but not so cluttered that it gets lost in details: The verifier needs tightly shaped inputs that let it answer specific questions; Patch and replan prompts need the right amount of evidence (too little and the planner will guess. Too much and it will waste cycles rewriting what already works.); Even tool outputs need to be treated carefully.

The quality of the system depends as much on how you use the context manager to shape and compress these contexts as on which model you use. The same model with well-optimized context will often outperform a larger one wrapped in noise.

Determinism, Cost, and Speed

Hybrid loops are gaining ground because they hit a sweet spot between predictability and power.

Determinism comes from the fact that the supervisor is real code and the models are constrained. There is no hidden control logic inside long prompts that change without anyone noticing.

Cost is contained because the number of heavy model calls is small. You are not paying for the model to re-read its entire context on every micro-decision.

Speed follows from the fact that the supervisor can aggressively fan out tool calls. There is no need for a single model to alternate between reading, thinking, acting, and reading again.

At the same time, you keep the human advantages of modern LLMs. They still do the hardest part: turning vague goals into workable plans.

The end result is a system that can do serious work, in real environments, at a cost and latency profile that leadership can live with, and with a level of traceability that risk and audit teams can understand.

Architecture That Thinks

If you accept this picture, it has a simple consequence for how you think about AI platforms.

Your AI infrastructure should not be a model with prompts; it must be a reasoning architecture. The Context Manager is its heart, pumping precisely the right information to the planner, supervisor, and verifiers. The hybrid loop is an architecture that does not just execute; it reasons.

For technical leaders, the important point is not the label. It is the underlying separation of concerns. If your systems still think in terms of stepwise chats with an “agent,” you will find yourself fighting the tools as models continue to improve. If your systems treat models as planners and judges wrapped in a deterministic execution fabric, you will be able to take advantage of that improvement with far less friction.

Hybrid loops are, in that sense, less a feature and more a sign that your architecture has caught up with what the models can actually do.

For More About Enterprise Context Management

Read our inaugural post: Your AI is Blind: Enterprise Context Management Arrives by and .

For more on an organizational architecture for context, check out The AI Context Skyscraper by .

Subscribe to the ECM Substack

Subscribe to the Enterprise Context Management Substack to track the evolving world of context management.

Subscribe now

Connect With Us!

Follow us. Because Context Matters :)

• AI One on LinkedIn

• Co-founder on Substack and LinkedIn

• Co-founder on Substack and LinkedIn

• The author of this article, , Chief AI Officer of AI One

• Editor and contributing writer to this Substack, (and follow Mark on LinkedIn or his Substack, .)

But while “context engineering” is an up-and-coming discipline in enterprises adopting AI, they’re often looking at the context the wrong way – as an engineering task – instead of a strategic business asset. Context is the very soul of the current AI age enterprise; the architecture of how you want your business to operate. Expecting engineers to engineer context alone is like a carpenter making cabinets without a blueprint for where they belong. Enterprise Context Management (ECM) is a rising enterprise software category that helps capture, curate, and deploy context at the right time and right place.

But to use context effectively, a new mental model is needed: The AI Context Skyscraper.

Context as Architecture

Instead of a single, undifferentiated block of text, imagine the context you provide to an LLM as a skyscraper, with each floor representing a different layer of information: corporate strategy on the top floor, customer interactions in the middle, and reusable recipes traveling by elevator from floor to floor.

The architecture of context is as important as the context itself. The position of each floor matters. The order, organization, and optimization of this context directly influences the LLM’s focus and the quality of its output.

Let’s explore this context skyscraper for FinCorp, a global financial services company, moving from the penthouse to the sub-basement with concrete examples.

The Penthouse: System Prompts

At the top of our skyscraper sits the system prompt. This is the most privileged real estate, and LLMs usually pay the most attention to what appears here, making it the core of the AI’s identity and objectives. This is where you define AI’s personality, style, and the fundamental principles that govern its behavior.

FinCorp’s system prompt might read:

“You are an assistant specializing in long-term financial advice and wealth management, maintaining a conservative approach to risk.

Ensure that all recommendations are tailored to each client’s specific goals, comply with SEC regulations, and align with our distinct investment principles and current market outlook. Prioritize accuracy over speed in every response, and always cite reliable sources.

After each recommendation, validate that the advice aligns with the client’s objectives, regulatory standards, and the latest market data; self-correct or escalate if there are any uncertainties. If uncertain about any information or recommendation, escalate the query to human oversight rather than offering speculative advice.”

System prompts, like penthouse residents, are your anchor tenants. They must be carefully curated, tested for suitability, then locked down. They represent your AI constitution. Your strategic north-star. Your principles. System prompts are treated with the reverence that position demands.

The Executive Suite: Subsystem Prompts

Just below the penthouse are subsystem prompts. This is your role identifier, the place where you specify the AI’s role for a specific set of tasks. It’s where the AI gets marching orders from your Chief Revenue Officer, Chief Marketing Officer, or Chief Financial Officer to align overall strategy with functional execution.

For example, FinCorp’s CFO might guide AI used to detect fraud with a subsystem prompt like this:

“You are a fraud research assistant responsible for assessing transaction patterns for unusual behavior, potential regulatory violations, and areas of uncertainty.

Immediately flag any anomalies that deviate by more than three standard deviations from established baselines. For each flagged anomaly, provide a one-line validation and indicate the next research step or correction if needed.

Before performing in-depth research on transactions involving high-risk jurisdictions or entities on watchlists, state the purpose and the minimum information you will use.”

Executive suite context prompts occupy the space just below the penthouse, and are owned by functional leaders at the highest level. They command significant attention from the LLM, making it a powerful lever for shaping behavior when the penthouse is off-limits.

The Upper Floors: Conversations, Workflow, History

As we descend into the upper floors, we encounter day-to-day interaction context. An example of context at this level are prior responses in an ongoing user conversation: every relevant question asked, every answer given, every clarification sought.

FinCorp’s conversational history might contain:

User: “Show me Q3 revenue for the EMEA region”

AI: “Q3 EMEA revenue was $45.2M, up 12% YoY”

User: “How does that compare to APAC?”

AI: “APAC Q3 revenue was $38.7M, up 8% YoY. EMEA outperformed APAC by $6.5M”

User: “What’s driving the EMEA growth?”

Conversational history is essential context, but it can become bloated quickly. This floor is a prime candidate for intelligent compression. You want to retain the essence of the conversation without overwhelming AI with every word that’s been exchanged, or including irrelevant distractions

Summarization techniques work well here, allowing you to preserve the narrative arc and key information while reducing the token footprint. The compressed version might become: “User is analyzing Q3 regional performance, specifically comparing EMEA ($45.2M, +12% YoY) vs APAC ($38.7M, +8% YoY), now investigating EMEA growth drivers.”

In addition to conversations, the upper floors also house the context of workflows. Workflows include the steps AI takes to reason through complex questions, showing which tools have been called, in what order, and with what parameters. Workflow context provides an audit trail and can be refined to build organizational wisdom for answering complex questions.

FinCorp’s question could require these steps:

Called Salesforce (Table=“revenue”; Filter=“Region”, EMEA, quarter=Q3)

Called Salesforce (Table=“revenue”; Filter=“Region”, APAC, quarter=Q3)

Called Database Calculate Variance(45.2, 38.7)

Next: Analyze Growth Drivers (Region=“EMEA”, quarter=“Q3”)

When crafting this context, prune irrelevant steps but maintain the sequence of actions. This floor provides the AI with a sense of what has been tried, what worked, and what didn’t, enabling more intelligent decision-making about next steps.

The Middle Floors: Results for Explainability

The middle section of our skyscraper houses some of the most critical and untouchable content: the raw output from relevant API calls, database queries, or data returned by external systems. This context provides the raw ingredients for explainability in AI. The ground truth. The audit trail. Result sets should rarely be optimized, compressed, or summarized by automated processes, only logged.

Raw output is the factual foundation upon which the AI’s reasoning is built, and any corruption of this data could lead to catastrophically wrong conclusions and the erosion of trust and verifiability in outcomes.

A database query might return:

{ “query_id”: “rev_q3_emea”,
  “timestamp”: “2024-10-15:23”,
  “results”: [
    {”country”: “Germany”, “revenue”: 18500000, “yoy”},
    {”country”: “UK”, “revenue”: 14200000, “yoy”},
    {”country”: “France”, “revenue”: 12500000, “yoy”}],
  “records”: 3}

The logging of the precise technical question asked ensures errors, bias, and hallucinations can be understood, audited, and fixed.

Along with the calls made to AI, we preserve the results.

Building on this raw data, the middle floor may also contain any summary tokens that might be provided by the AI’s reasoning or chain of thought: its step-by-step progression of how it arrived at a recommendation. For example, FinCorp’s reasoning chain might have been:

User asked: “EMEA growth drivers”

Retrieved: Country Level (Germany (+15%), UK (+9%), France (+11%))

Observation: Germany represents 41% of EMEA revenue and has highest growth rate

Prediction: This suggests Germany is the primary growth driver

Next: Explore what changed in the German market in Q3

The data from this floor should remain largely untouched, serving as the intellectual scaffolding of the AI’s decision-making process. Storing questions, results, and reasoning chains ensures you can always show your work and demonstrate how the AI arrived at its conclusions.

The Lower Floors: Retrieved Context and Governance

As we move into the lower floors, we encounter the next set of context for the question: the information retrieved from knowledge bases, vector databases, or other sources to help answer the current query.

When investigating EMEA growth drivers, the system might extract key facts from a knowledge base:

Germany Q3 2024: Launched new enterprise product tier in July

UK Q3 2024: Faced increased competition from local fintech startups

France Q3 2024: Expanded partnership with major retail bank, added 150 enterprise customers

EMEA Market Report Q3: Manufacturing sector had 18% increase in spending

Rather than simply dumping results from a vector similarity search using RAG, a more sophisticated approach involves a “Context Loop”, which uses intelligent chunking and multiple similarity search passes on the raw underlying documents and data-sources, to explore the best match, drill down, and follow threads of relevance. Instead of taking a single retrieval pass, the Context Loop successively determines the best, most relevant, context. Context looping is particularly important when extracting context from numerous large documents, extensive database schemas, or complex knowledge graphs.

Below Context Loop-extracted context, we establish the guardrails, the hard rules, and the constraints that govern the AI’s behavior. These are non-negotiable boundaries: what data the AI cannot access, what actions it cannot take, and what topics it must avoid:

NEVER access or display individual customer names or personal identifiers

NEVER execute queries against production databases, only read replicas

NEVER recommend actions that would violate GDPR or data residency requirements

NEVER process or display salary information for identifiable individuals

MUST reject any request to bypass audit logging

MUST escalate to human oversight for any transaction >$1M

These guardrails are strictly enforced and should never be compressed or optimized away. They are the safety systems of your AI, the circuit breakers that prevent catastrophic failures.

Adjacent to the guardrails, we have conduct and expectations—a softer set of guidelines that shape the AI’s personality and communication style:

Maintain a professional, formal tone appropriate for financial services

When presenting numerical data, always include units and time periods

Acknowledge uncertainty rather than presenting speculation as fact

Provide context for percentages (e.g., “15% growth on a base of $18.5M”)

Use inclusive language and avoid regional bias when comparing markets

Cite data sources for all quantitative claims

Unlike the hard guardrails, these are more like cultural norms, shaping how the AI presents itself and interacts with users.

The Basement: Playbooks and Multi-Agent Integration

The basement of our skyscraper houses the playbook, a truly innovative feature. This is an LLM-populated guide that captures the accumulated wisdom from previous executions of similar workflows. Every time the AI successfully completes a task, it can contribute to this playbook, noting which strategies worked well and which approaches led to dead ends.

PLAYBOOK - Revenue Analysis Workflows

Pattern: Regional growth investigation
- Success strategy: Start with country-level breakdown 
- Use: regional_sales

Pattern: EMEA analysis
- Avoid: Don’t rely on revenue table - it excludes partner channel sales
- Use: combined_revenue_view

Pattern: Growth driver identification
- Success strategy: Check for product launches
- Success strategy: Check for partnership announcements

The playbook represents organizational learning at scale, turning every AI interaction into a source of competitive advantage. It’s how your AI gets smarter over time through the accumulation of procedural knowledge. When faced with a new instance of a familiar problem, the AI can consult the playbook to see what has worked before, dramatically improving its efficiency and success rate.

This floor should be carefully curated but not aggressively compressed. The insights here are valuable precisely because they contain nuance and detail. Strip that away, and you lose the very knowledge you’re trying to preserve.

Finally, we have space for input from other agentic frameworks. In a multi-agent system, where different AI agents handle different aspects of a complex workflow, this floor provides a clean integration point. It’s where Agent A can pass context and results to Agent B, enabling sophisticated orchestration without contaminating the other floors of the skyscraper.

This separation is crucial for maintaining clarity and debuggability in complex systems. When something goes wrong, you need to be able to trace the flow of information between agents, and having a dedicated floor for inter-agent communication makes that possible.

The Context Foundation Meets the Task At Hand

Underpinning all the prior context sits the prompt itself: the specific instructions for the current task.

First should come the original question that we are seeking an answer to. This is the user’s initial query that triggers the entire workflow. Even as the AI moves through multiple steps in an agentic loop, calling various tools and processing information, this original question serves as a north star.

“What’s driving EMEA growth?”

It’s the constant reference point that ensures the AI doesn’t drift off course, doesn’t get lost in the weeds of intermediate processing, and ultimately delivers an answer to what the user actually wanted to know.

Finally, we reach the current task. This is what the AI is being asked to do right now, at this moment. It needs to be clear, unambiguous, and immediately actionable.

“Using the country-level revenue data retrieved from the database, identify the top 3 contributors to EMEA growth in Q3. For each country, calculate its contribution to the overall regional growth and identify any significant product or customer segment changes. Present findings in order of impact magnitude.”

This is the AI’s current focus, the question it’s actively working to answer. It is at the bottom of the prompt due to something known as recency bias, where LLMs pay a little more attention to the most recent information it’s been given.

Some models will even improve their response if this is underlined with one repeated strata of instruction such as:

“Provide a ranked list of the top 3 EMEA countries driving Q3 growth, with each country’s % contribution and key product or segment changes. No commentary or extra text.”

This question-answering is where the context rubber meets the road, and it takes careful interplay with the LLM to keep it on task and context-aware.

The Fatal Flaw of One-Size-Fits-All Optimization

The tour of our skyscraper reveals the fundamental flaw in current context management: a one-size-fits-all approach to optimization. Most optimization techniques treat the entire context window as a single, undifferentiated mass. They apply the same algorithm to every floor of the skyscraper, often all at the same time.

This is architectural malpractice. It’s like trying to renovate a building by applying the same design to the penthouse, the boiler room, and every office in between. You wouldn’t insulate your lobby the same way you insulate your roof, and you shouldn’t optimize your tool results the same way you optimize your conversational history.

The list of optimization techniques available to apply to each floor is extensive, including some such as:

• Query-focused Summarization

• GEPA (GEnetic-PAreto Optimization)

• ACE (Agentic Context Engineering)

• Chain of Verification

• Targeted Truncation and redaction

• Self refine/Critique re-write

• Instruction decomposition

• Context salience labelling

Each floor has different requirements, different sensitivities, different roles in the overall structure. The system prompt needs to be protected and optimized once. Tool results should never be compressed. Conversational history can and should be summarized. The playbook needs careful curation. Guardrails must remain intact.

When you apply a blanket optimization strategy across the entire context window, each floor inherits the disadvantages of that approach. You end up compromising the integrity of the entire building, losing critical details like the one-time bonus in the France data while wasting tokens on redundant conversational history.

The Power of Floor-Specific Strategies

The alternative is to manage each floor independently, applying the optimization strategy that makes sense for that particular type of content, and for the LLM that you are targeting. This might mean using one compression technique for conversational history, a different approach for workflow history, and no compression at all for tool results and reasoning chains. A different LLM will require different optimizations for each floor, and even potentially move some floors around.

This granular control is not merely academic; it is a practical necessity for building robust, reliable, enterprise-grade AI systems. It allows you to maximize the effective use of your context window, preserving the most important information while still fitting within token limits. Flexible meta-optimization architectures enable switching from one LLM to another easily, eliminating single vendor dependencies. It enables you to maintain explainability and traceability where required, while still achieving efficiency where possible.

Moreover, this approach is fundamentally more flexible. As new optimization techniques emerge, you can adopt them selectively, applying them to the floors where they make sense without disrupting the rest of your carefully constructed context architecture.

Sovereign Context: Taking Back Control

The most important implication of the context skyscraper model is the concept of sovereign context. When you view context as a complex, multi-layered structure that requires sophisticated management, it becomes clear that you can’t afford to outsource this responsibility to third-party AI providers, or allow context to be applied in an ad-hoc manner.

Your context is not just data. It’s the distilled essence of your business, your processes, your institutional knowledge. It’s the competitive advantage that makes your AI uniquely valuable to your organization. When you hand control of that context over to a SaaS provider or a frontier LLM provider, you’re not just creating a security risk, you’re giving away strategic control.

Consider our FinCorp example. The playbook contains hard-won knowledge about data quirks, successful analysis patterns, and domain-specific insights. The guardrails encode your risk tolerance and regulatory obligations. The conversational history reveals your analysts’ thinking patterns and priorities. This is not generic, commoditized information; this is your institutional intelligence.

Sovereign context means building, managing, and owning your context skyscrapers. It means having the flexibility to fine-tune your context for specific tasks without being locked into a vendor’s optimization strategy. It means ensuring the security and privacy of your data by keeping it under your own roof.

In the same way that it’s unreasonable to expect a standardized skyscraper to fit into every city’s skyline, it is not realistic to expect a context window optimized for a specific model to generate the same results when thrown into a different model. Different LLMs and tools prefer different skyscraper “shapes” and techniques in order to provide the most accurate results.

The risk of optimizing a context window to work only for one model is to create a certain level of single vendor dependence that becomes undesirable in the future. The benefit of using a flexible, floor-based context optimization strategy is that it is trivial to create multi-vendor versions of everything, so you always have the right shaped building for the city you are building in.

Enterprises that treat their context as a first-class asset, as valuable as their data, code, and other intellectual property, will be the ones to thrive in the AI era.

Build Your Own Context Skyscraper

As you embark on your enterprise AI journey, adopt the mental model of the Context Skyscraper. Don’t think of context as a blob of text, think of it as a carefully architected building, with each floor serving a specific purpose and requiring a tailored management approach.

Build your skyscraper with intention. Understand what belongs on each floor. Apply the right optimization strategies to each level. Protect the floors that need protection, and optimize the ones that can afford it. And above all, never give away the keys to your building.

The context creation loop is where the real value of enterprise AI is unlocked: the continuous, iterative process of extracting context, managing it through your skyscraper, and using it to drive intelligent workflows. It’s the difference between AI that merely responds to prompts, and AI that truly understands and advances your business objectives.

The skyscraper is yours to build. Build it well, and you will unlock the true power of enterprise AI.