Fresh new look, same great taste!

We’ve refreshed the design but kept the useful content you’re used to.

We’re looking to add new sections over the coming weeks, if there’s anything you’d like to see, hit reply and let us know.

IN CASE YOU MISSED IT

Hidden gems

MCP learning path from fundamentals to conformance

Across 17 hands-on lessons, the course covers MCP clients, servers, transports, security, authorization, gateways, registries, reliability, and conformance, using the 2026-07-28 specification throughout.

Security analysis of inference engine exploits

Using a real vLLM arbitrary-code-execution bug as evidence, it examines how malicious model outputs could exploit inference parsers, compromise GPU hosts, persist through tool use, and how operators can reduce exposure.

Microharness for persistent agents

Built around a continuous self-guided thought loop, Headlong combines Bash-based tools, persistent trajectories, memory compaction, and sandboxing to test agents that keep working between external interactions.

Visual review tool for AI-generated files

Built for HTML, Markdown, and localhost pages, it lets you edit content visually, attach comments to specific elements, and send consolidated feedback back to your coding agent.

AGNTCON + MCPCON EUROPE

36 hours live from Amsterdam

Get the coffee and stroopwafels ready - we’re going live from AGNTCon + MCPCon Europe for 36 hours straight, bringing you the talks, speaker interviews, expo floor conversations, demos, after-hours chats and plenty that won’t appear on the official agenda.

We’ll be moving around the conference, catching speakers and builders between sessions, following the conversations happening across RAI Amsterdam and bringing questions from the livestream chat into the event.

Drop in for a keynote, stick around for the aftershow, or see how long you can stay with us through the night shift.

Watch from home: Tune in across September 17–18 for the full 36-hour broadcast. Register free

Want to be there in person? Join us in Amsterdam and come say hi. Early Bird tickets are available until September 4. Get your ticket here

AAIF COMMUNITY

How predictive analytics stops budget overruns before they happen

A model change can push token usage up 8x, while retry-heavy agent loops can burn through budget before conventional billing alerts catch the problem. The discussion looks at how FinOps can move closer to the engineering workflow as AI usage scales.

• Predictive budgets can flag teams or service accounts heading far beyond quota early in the month, giving engineers time to adjust rather than simply hitting a hard stop.

• FinOps checks can be built into system prompts, pull requests, and deployment workflows, surfacing policy, model, and infrastructure costs as code changes.

• Some workflows may be better served by rules, retrieval, or conventional ML, with LLMs reserved for the steps that genuinely need generation.

The core idea is to make cost an engineering signal that appears early enough to shape architecture, model choice, and day-to-day decisions.

Video · Spotify · Apple

The Winchester Mystery House Problem in AI development

A workflow that cost $1 per 1,000 records at 90% accuracy reached 95% accuracy while costing less, after an optimizer moved 75% of the work out of the model and into code. That result sits inside a broader discussion about when agentic behavior should become a defined workflow.

• Models increasingly inherit assumptions from the coding harnesses they were trained around, which can make custom harnesses harder to build.

• Repeated agent tasks can often be “crystallized” into smaller models or conventional code once the process is understood.

• DSPy separates task definitions from implementation, allowing prompts, models, and even harness code to be optimized without redefining the task.

The practical direction is toward identifying which tasks still need open-ended agents and which are mature enough to become cheaper, more reliable workflows.

Video · Spotify · Apple

MCP 2026–07–28: Stateless core, enterprise authorization, and SDK betas

Sticky sessions and shared session stores are no longer required under MCP’s 2026-07-28 specification, removing a major obstacle to scaling remote servers across multiple instances. The same release cycle also brings centralized enterprise authorization and a more predictable protocol lifecycle.

• The stateless core makes every request self-describing, enabling standard round-robin load balancing and multi-round-trip interactions without persistent sessions.

• Enterprise-managed authorization uses ID-JAG and token exchange to connect existing SSO, roles, and audit controls to MCP servers.

• Formal deprecation windows, conformance testing, and beta SDKs give teams a clearer path for testing and migrating between protocol versions.

The changes move MCP closer to infrastructure that teams can scale, govern, and upgrade using established production patterns.

Read the blog

Building a read-only multi-cloud Kubernetes inventory with goose

A read-only inventory agent can still have more authority than the task requires if its shell or cloud credentials are too broad. This design walkthrough uses goose to show how OKE and AKS inventory could be constrained without giving the model write-capable access, while clearly separating planned behavior from validated results.

• The prototype combines read-only cloud identities, supervised shell access, and two exact inventory commands, while acknowledging that recipe instructions are not a technical allowlist.

• Structured output preserves partial failures, unknown values, warnings, and provenance, with independent logs needed to verify what actually ran.

• A stronger production boundary would replace the general shell with narrow MCP tools that expose only the required read operations.

The main design lesson is to keep authorization outside the model and treat prompts, approvals, and containers as supporting controls rather than security boundaries.

Read the blog

Where A2A fits in the open agent ecosystem

A restaurant manager agent can use one protocol to delegate an urgent order, another to read inventory, and others to structure checkout and verify payment authorization. The piece uses that flow to show where A2A fits alongside MCP, UCP, AP2, and existing web standards.

• A2A gives independent agents a shared contract for discovery, delegation, stateful tasks, progress updates, and artifacts without exposing their internal implementation.

• MCP covers tool and context access, while A2A handles collaboration between autonomous endpoints with their own execution logic.

• Extensions let domain protocols such as UCP and AP2 add commerce-specific data and authorization without expanding A2A’s core.

The architecture depends on clear protocol boundaries, so each layer can specialize while remaining composable with the others.

Read the blog

COME AND CONNECT

IN-PERSON EVENTS

• Silicon Valley - August 27

• Denver - August 27

• Melbourne - September 2

• Shanghai - September 5

• AGNTCon + MCPCon China - September 6

• Toronto - September 10

• AGNTCon + MCPCon Japan - September 10

• Pune, India - September 12

• New York - September 15

• San Francisco, Voice Agents Forum - September 16

• AGNTCon + MCPCon Europe - September 17

Find your city here, or start a chapter if there isn’t one yet.

JOIN FROM ANYWHERE

VIRTUAL EVENTS

• Reading Group: Prompt Injection as Role Confusion - August 27

• 36 Hours Live from AGNTCon + MCPCon Europe - September 17 - 18

MEME OF THE WEEK

Thanks for reading. See you on Discord, YouTube, GitHub, X, and LinkedIn.

Share

HOT TAKE

Taste test

When agents can write the code, engineering advantage shifts from implementation to judgment.

What becomes more valuable: Execution or Taste

EXECUTION or TASTE

LAST WEEK’S TAKE

Keeping it fresh

You don’t trust your agent to mark its own homework, preferring a fresh context to verify the code.

READING GROUP • NEW DATE: AUGUST 27

When untrusted text looks like an instruction

An agent pulls in content from a webpage, tool, or document, and the model treats that untrusted text as though it were an instruction from someone it should obey.

Prompt Injection as Role Confusion argues that this failure to distinguish between different roles may help explain why prompt injection remains such a difficult security problem.

Join the AAIF reading group on August 27 to discuss the paper, its arguments, and what this framing could mean for building safer agents and systems.

Register for the reading group

If you have time, read the paper beforehand and bring your questions. If not, come along and work through it with the group.

HIDDEN GEMS

Curated finds to help you stay ahead

Stress-testing self-improving agents

Re-evaluating memory-based agents across multiple runs and shuffled task orders, the paper finds that apparent improvement can depend heavily on evaluation noise, sequencing, and underspecified tasks.

Contextual feedback tool for coding agents

Capturing selected text, screenshots, web elements, and voice notes with their source context, Remarc hands structured feedback to coding agents through MCP, clipboard export, or webhooks.

Enterprise agent workflow design

Using a churn-detection system as the case study, the piece shows how to split work between code, embeddings, agents, and humans while keeping decisions verifiable and feedback useful.

Agent security beyond model safeguards

Framing agents as privileged identities, the piece connects model behavior with credentials, tools, and production access, arguing for security controls that account for changing model and deployment risk.

JOB OF THE WEEK

Senior Applied AI Engineer // Forter // London, UK

Forter is hiring a Senior Applied AI Engineer to work directly with business teams on production AI systems, from scoping opportunities through deployment and adoption. The role spans agents, MCP, RAG, evaluation frameworks, guardrails, and internal AI tooling.

Responsibilities

• Identify and prioritize high-impact AI opportunities across business teams

• Own AI solutions from requirements and development through deployment and adoption

• Build on shared infrastructure including MCP servers, RAG systems, eval frameworks, and guardrails

• Measure adoption, time saved, process improvements, and other outcomes

Requirements

• 7+ years of software engineering experience shipping production systems

• Experience working directly with non-technical teams on technology projects

• Strong product judgment, communication skills, and ability to work independently

• Experience with LLMs, prompt engineering, RAG, agents, and tool use

  ---

MLOPS COMMUNITY

How to delegate to an agent like you would an employee?

An agent can rebase a PR, chase failing CI, respond to review comments, and keep checking until every test is green while you work on something else. The discussion gets into what happens when computer use moves from occasional assistance to something closer to continuous delegation.

• Computer use lets agents operate across logged-in apps, forms, dashboards, notebooks, and spreadsheets, including workflows where building or maintaining an API integration would be more effort than simply using the interface.

• That autonomy needs controls at several layers, including sandbox restrictions, MCP permissions, classifiers, and separate model review before potentially sensitive actions are allowed through.

• One of the harder open questions is when an agent should keep trying and when it should stop and ask for help. Too little persistence wastes the delegation, while too much can create its own problems.

As agents get better at doing the work, deciding when they should hand control back may become one of the more important parts of designing them.

Video · Spotify · Apple

The affordable AI agents

A 5,000-developer engineering team saw its AI bill jump 5x in a month after autonomous coding agents started burning through usage-based credits in background debug loops. The response was a wider look at where managed APIs, self-hosted models, and local inference make financial sense.

• Usage-based pricing changes the economics of agentic workloads because every context load, tool call, and repeated loop carries a direct cost, making model choice and context size infrastructure concerns rather than small optimization details.

• Self-hosting can cut per-query costs for predictable, high-volume workloads, but GPU capacity, GKE operations, and specialist engineering time can erase much of the apparent saving.

• Local inference avoids API charges but introduces its own tradeoffs around memory limits, prefill latency, hardware performance, maintenance, and loss of centralized telemetry.

The practical answer is likely workload routing, with different models and hosting approaches chosen according to task complexity, volume, latency, and operational cost.

Read the blog

Putting a front door on local AI with agentgateway

One unauthenticated request burned 1,302 tokens of GPU time, and the model server had no record of who sent it. That is the kind of problem that appears once a local model stops being a personal experiment and starts serving other users and agents.

• Putting an AI-aware gateway in front of Ollama adds authentication, stable model names, token accounting, and a consistent endpoint without changing the models underneath.

• Token-based limits can stop an agent loop consuming unlimited shared capacity, although the standalone setup uses one shared budget rather than separate quotas for each API key.

• Health-based routing can move traffic to a backup model when the primary fails, but the first failed request can still reach the client before the unhealthy target is evicted.

Running the model is only part of the job once other people and agents depend on it.

Read the blog

Building a governed MCP platform with agentgateway

A plausible-looking authorization policy in the wrong place can leave a tool completely unenforced. That was one of several issues exposed while building a local reference platform for identity, MCP authorization, model failover, and observability around a shared agent gateway.

• MCP authorization needs to reach the tool-call boundary, not stop at server access. The test setup filtered tool discovery by role and tenant, then separately rejected unauthorized writes and direct cross-tenant calls.

• Runtime testing caught problems that static configuration review missed, including misplaced policies, unsafe tenant selection, failover that required health-based eviction, and an identity-provider startup race.

• Observability was tested behaviorally too, with authenticated requests producing expected metrics and traces rather than treating a provisioned dashboard as proof that telemetry worked.

For agent infrastructure, configured controls only become meaningful when tests prove they allow and deny the right things.

Read the blog

IN-PERSON EVENTS

• Silicon Valley - August 27

• Denver - August 27

• Melbourne - September 2

• Shanghai - September 5

• AGNTCon + MCPCon China - September 6

• Dallas - September 9

• Toronto - September 10

• AGNTCon + MCPCon Japan - September 10

• Pune, India - September 12

• New York - September 15

• San Francisco, Voice Agents Forum - September 16

• AGNTCon + MCPCon Europe - September 17

VIRTUAL EVENTS

• Coding Agents Lunch & Learn Session 22 - August 21

• Reading Group: Prompt Injection as Role Confusion - August 27

  ---

MEME OF THE WEEK

ML CONFESSIONS

The JSON Supremacy

I spent three weeks improving a RAG system that kept missing obvious answers. New embeddings, different chunk sizes, reranking, hybrid search. Recall went up a bit. Users kept complaining.

Then I found an old debug branch still live in production. For the 40 most common questions, it skipped retrieval and returned a hand-maintained answer from a JSON file someone had built months earlier and forgotten to remove.

Those 40 questions had the highest satisfaction scores of anything in the product.

We fixed the retrieval system and left the JSON file running alongside it, because nothing we built matched what a person had just typed out by hand.

Share your confession here.

Share

HOT TAKE

Fresh Pair of AIs

The best verifier is an agent that did not write the code.

Who should check the work? Same context or Fresh context

SAME or FRESH

LAST WEEK’S TAKE

Rules of engagement

Close, but more of you back writing better rules over upgrading your model.

VOICE AGENTS FORUM // SAN FRANCISCO

Building voice agents for production

Latency, interruptions, handoff, evaluation, observability, reliability. Voice agents bring a set of engineering and operational problems that only become more important once people start using them.

Join us at Digital Jungle SF on September 16 for a day with engineers, founders, researchers, and operators building and deploying voice systems. Compare how teams are approaching the stack, where technical choices create tradeoffs, and what to consider as voice agents move from prototypes into production.

Early Bird tickets end September 1.

Register now

HIDDEN GEMS

Curated finds to help you stay ahead

AI code review orchestration guide

Seven specialized agents, coordinated through a CI-native plugin architecture, provide a practical design for scaling automated reviews while controlling noise, duplication, severity, cost, and failures.

Static retrieval training and benchmark

Training on 660 million pairs and aggressive quantization produce an 8 MB retrieval model that can embed English Wikipedia on a laptop in under eight minutes.

Coding agent harness training guide

Running each rollout inside the deployed harness lets TRL train on captured model calls and hidden-test rewards, preserving the agent’s genuine tool loop rather than recreating it.

Reproducible evals for SRE MCP servers

Deterministic telemetry, isolated agent runs, and blind scoring form an open evaluation framework for comparing MCP tools against direct SQL across realistic incident-investigation scenarios.

JOB OF THE WEEK

Senior Machine Learning Engineer // Thorn // Remote, US

Thorn is hiring a Senior ML Engineer to build, deploy, and maintain models that help identify victims, detect abusive content and behavior, and support the removal of child sexual abuse material. The role covers model development, evaluation, data labeling, and production infrastructure.

Responsibilities

• Own end-to-end development of production ML systems and algorithms

• Define data sourcing, labeling requirements, and evaluation processes

• Deploy and maintain models with product engineering teams

• Present technical work and collaborate with external partners

Requirements

• 5+ years of experience in ML or AI

• Experience building production ML systems and pipelines

• Production experience with computer vision or NLP

• Clear communication and willingness to work in online safety

  ---

LUNCH & LEARN · SESSION 21

856 tools, 103 servers, one quality score

Tomorrow, Friday 14 August, 16:00-17:00 UTC. Free and virtual.

Om Shree is presenting TDQS (Tool Definition Quality Score), a framework for measuring MCP tool quality that came out of a study of 856 tools across 103 live MCP servers. He’ll cover what separates a tool definition an agent can use reliably from one that produces confusing behavior, and what to check in your own tools. Community discussion on the MCP ecosystem follows the talk.

Register here

The last session ran on a related theme. Leo Walker walked through Arcade’s ToolBench benchmark for grading MCP servers, and why a strong compliance grade still doesn’t guarantee a clean connection. John Dellenbaugh demoed the stateless protocol change that lets you scale MCP servers across instances without a sticky gateway or a session store, and Ben Kearns pushed back on what happens to authorization when a client carries its own identifier.

Notes from the session

MLOPS COMMUNITY

Why Supervising AI Code Is Harder Than Watching a Car Drive Itself

Supervising a coding agent may be more cognitively demanding than watching a self-driving car because its intent and acceptable end state are far harder to observe.

• Self-driving works within defined inputs, goals, and physical constraints; software development is open-ended, with much of its intent missing from the code.

• Graph-based workflows can run independent agent tasks in parallel, then use fresh context to verify and merge results.

• Growing skill libraries need evaluation, pruning, and reliable selection across models.

Better agents still demand better ways to supervise their reasoning.

Video · Spotify

Agent evolutions: Stop guessing the design - evolve it

The winning design cut CLI startup from 142.0 ms to 98.6 ms, yet it did not exist when the search began.

• A genetic coding-agent loop generates diverse approaches, keeps strong performers, and mutates or combines them across generations.

• Binary gates exclude broken variants, while a frozen numeric rubric prevents the scoring criteria shifting after results appear.

• File-based results preserve audit trails, survive crashes, and keep parallel sub-agents isolated.

Measured evolution can find designs that intuition and one-shot sampling miss.

Read the blog

The enterprise MCP scaling paradox: why every agent needs a tailored backend

Successful MCP adoption could leave an enterprise running hundreds of servers, each adding compute, maintenance, security review, and on-call responsibilities.

• Broad vendor tool catalogs increase token use, latency, and attack surface while exposing far more capabilities than individual agents need.

• Internal servers fragment identity, deployment, and telemetry unless teams share a clear platform contract.

• Gateways, generated tool definitions, and agent-specific backends each reduce some duplication but introduce their own tradeoffs.

MCP scale depends on governing the full estate before server sprawl sets in.

Read the blog

From Skills and Tools to Portable Agent Plugins

A deployment assistant combining reusable instructions and live tools can now travel across compatible agent clients without being repackaged for each one.

• Agent Plugins 1.0 places native Agent Skills and MCP configuration inside one predictable directory structure and manifest.

• Client-specific features remain in namespaced folders that unsupported clients can safely ignore.

• Installation, permissions, trust, distribution, and runtime behavior stay under each client’s control rather than being forced into the shared format.

The result is a narrow portability layer that preserves room for clients to differ.

Read the blog

IN-PERSON EVENTS

• New York - August 14

• Silicon Valley - August 27

• Melbourne - September 2

• Shanghai - September 5

• AGNTCon + MCPCon China - September 6

• Toronto - September 10

• AGNTCon + MCPCon Japan - September 10

• Pune, India - September 12

• New York - September 15

• San Francisco, Voice Agents Forum - September 16

VIRTUAL EVENTS

• Coding Agents Lunch & Learn Session 21 - August 14

• Reading Group: Prompt Injection as Role Confusion - August 18

  ---

MEME OF THE WEEK

ML CONFESSIONS

Predicting the Past

I got 0.94 AUC on a churn model and presented it twice. The second time someone asked what account_status_updated_at was doing in the feature list. It was the timestamp on the cancellation record.

I said I’d double-check and moved to the next slide. Rebuilt it over the weekend without that column and got 0.71. Sent the new numbers round on Monday with no explanation. Nobody replied.

Share your confession here.

Share

Six cities, seven events, 475+ attendees, and countless cake.

We pulled together some of the numbers, event recaps, community posts, and coverage in one place so you don’t have to go hunting through LinkedIn. Have a look.

HOT TAKE

Time is money

Most teams paying for a frontier model would get more from an hour spent on their AGENTS.md.

What moves the needle more? Rules or Model

RULES or MODEL

LAST WEEK’S TAKE

What’s the frequency, Kenneth?

The consequence of asking what should guardrails minimize: a clean sweep.

FREE VIRTUAL EVENT - TODAY

What breaks when you run MCP in production

Speakers from eight teams share what holds up once MCP has to handle production security, observability, governance, and multiple agents.

Expect live demos and practical lessons, including source-level findings from 100 MCP servers, agent swarm orchestration, an MCP-powered observability stack, and what the 7-28 spec changes and how to migrate.

Happening today, 15:00-19:00 UTC / 08:00-12:00 PDT. Free and virtual.

Register now

HIDDEN GEMS

Curated finds to help you stay ahead

Readable Python coding agent

Built around a portable, provider-neutral harness, Tau reads and edits files, runs shell commands, preserves sessions, and provides a compact reference implementation for understanding coding-agent architecture without a large production codebase.

Tunix high-throughput agentic RL framework

Combining asynchronous trajectory collection, barrier-free training pipelines, modular agent and environment APIs, and lightweight profiling, Tunix helps JAX/TPU teams reduce idle time and identify bottlenecks in multi-turn RL workloads.

Agent developer workspace architecture

Built around Coder-based workspaces, Monaco’s setup combines isolated VMs, cloned databases, prebuilt AMIs, private networking, and controlled credentials to support multiple coding agents working in parallel.

Random-vector reasoning experiment

Testing Qwen3 models across arithmetic, planning, legal reasoning, and debugging tasks, the study measures whether noisy embedding prefixes and plurality voting improve small-model accuracy without fine-tuning or larger hardware.

JOB OF THE WEEK

Agentic AI Platform Engineer // On // London, UK

On’s newly established AI and ML Platform team is hiring an engineer to build and operate production agentic infrastructure, including gateways, evaluation systems, observability, and governance, while supporting product teams adopting agent-based workflows across the business.

Responsibilities

• Architect and operate secure, observable infrastructure for production agent systems.

• Build evaluation frameworks, routing layers, gateways, and monitoring capabilities.

• Guide product teams on scalable architecture for agent-based workflows.

• Establish engineering standards and assess tools for shared platform use.

Requirements

• Three-plus years’ software engineering experience for mid-level applicants.

• Production experience building agentic AI or LLM platform components.

• Experience with cloud infrastructure, preferably GCP, and agent frameworks.

• Practical knowledge of MCP, vector databases, evaluation, and observability.

  ---

MLOPS COMMUNITY

Why your AI bill will double before it gets better

LLM costs are no longer falling fast enough to offset exploding usage, and hidden premiums can turn a viable feature into an expensive mistake.

• Cost calculators built into evaluation workflows can flag unscalable experiments before teams spend weeks prototyping.

• Token telemetry by service and feature makes spikes, model choices, and experiment costs visible.

• Optimization needs researchers, engineers, and finance working from the same latency, quality, and cost data.

Teams that price AI early can ship faster and avoid learning the economics after launch.

Video · Spotify · Apple

The EU AI Act and the new rules for building AI agents

An agent can issue a refund in seconds, but the company may later need to show which model ran, what data it used, which policy permitted the payment, and where human intervention was available.

• Risk classification follows the agent’s intended use, affected users, data access, and tools rather than model capability alone.

• Teams remain responsible for the assembled system, including access controls, human approvals, version records, and logs of tool requests and results.

• Material changes to its purpose, permissions, or models may require a fresh assessment.

Traceability and control need to be part of the production design before deployment.

Read the blog

AI music needs more knobs: YouAndOrchestra makes conducting agents a compositional exercise

One-shot generation can produce a polished track, but it gives creators few ways to correct one weak section without starting again.

• YouAndOrchestra assigns composition and critique to specialist agents coordinated through an iterative conductor loop.

• Users can intervene globally, at section level, or down to a bar, beat, or instrument.

• Every note carries provenance, showing which agent role proposed it, which musical rule it followed, what feedback redirected it, and how the decision was applied.

• The work is forkable and versioned, so earlier states remain available.

The project offers a useful model for agent systems where people need precise control over evolving work.

Read the blog

Agent Levers: A plan-do-check-act loop that makes coding agents finish what they start

A coding agent can pass every test and still build the wrong feature. Agent Levers puts the workflow on disk so intent, verification, and stopping conditions remain visible.

• Tasks move through a plan, do, check, and act cycle with explicit acceptance criteria.

• Verifiers run from a fresh shell, while iteration budgets stop failed loops from expanding the diff.

• Recurring mistakes become proposed rules in AGENTS.md for future sessions.

The result is less supervision without surrendering control over what gets shipped.

Read the blog

IN-PERSON EVENTS

• New York - August 14

• Toronto - September 10

• San Francisco, Voice Agents Forum - September 16

VIRTUAL EVENTS

• Virtual MCP event - August 6

  ---

MEME OF THE WEEK

ML CONFESSIONS

DAG Another Day

Three years ago someone senior pushed for us to move every training pipeline onto an orchestration framework they’d used at their last company. I argued against it in three separate meetings. What I said was that it would add operational burden we didn’t have the headcount for, and that we’d be rewriting DAG logic that already worked fine.

What I thought was that I’d just come off a fortnight of a different migration and could not face learning another set of abstractions.

We didn’t move. About a year later the main maintainer stepped back and the company behind it got bought. The org that had standardized on it spent most of last year moving off. I’ve still got the Slack message from one of their engineers saying I’d called it early.

Nobody has asked me to explain the reasoning. I get pulled into architecture reviews for this now, and there’s one next week about an agent framework. I’ve read maybe half the docs and I already know what I’m going to say.

Share your confession here.

Share

HOT TAKE

Frequent f-Liar

Low-value hallucinations may be cheaper to tolerate than prevent.

What should guardrails minimize: Frequency or Consequence?

FREQUENCY or CONSEQUENCE

LAST WEEK’S TAKE

Tough Call

Models aren’t the only ones who struggle to make a call - a dead 50/50 on whether the server or client should own tool overload.

HIDDEN GEMS

Curated finds to help you stay ahead

Stateless MCP Scaling Demo

Using a shopping-cart MCP server, the demo compares sticky-session and stateless deployments, with Cloud Run logs, pod recycling, and concurrent requests showing how client-carried state enables round-robin scaling.

Coding Agent Harness Efficiency Benchmark

Holding DeepSeek V4 Flash constant, the benchmark compares Pi, OpenCode, Claude Code, and Nanocoder across bug fixes, finding similar graded quality but substantial differences in tokens, tool calls, and completion time.

Pairwise SRE Agent Evaluation Method

Comparing similar incident-investigation traces with an LLM judge and converting pairwise wins into Elo scores provides an unsupervised signal for tracking SRE agent quality without immediate ground truth.

SkyPilot Multi-Cloud GPU Platform Launch

Combining fragmented GPU capacity across clouds and clusters, the platform provides fleet management, governance, and support for pretraining, post-training, multi-cluster serving, and sandboxes without replacing existing SkyPilot workflows.

MCP 7-28

Five Guides to the New Specification

The MCP community marked the 7-28 release with launch parties around the world this week. To help teams understand what changed and plan their transition, we’ve collected five AAIF blogs covering migration, stateless architecture, platform governance, deprecation, authorization, and long-running tasks.

MCP 2026-07-28: What’s Changing and How to Migrate

A practical migration guide to the release’s stateless core, Tasks and MCP Apps extensions, authorization changes, and deprecations, with concrete checks for MCP client and server maintainers.

Migrate Sessions to Stateless Requests with MCP 2026-07-28

A code-led TypeScript migration guide showing how to replace session IDs and transport registries with per-request server instances, explicit state handling, standard load balancing, and controlled legacy-client support.

7-28 Hands MCP to Platform Teams

A platform engineering guide to the release’s stateless architecture, authorization boundaries, schema validation, and observability changes, with practical steps for governing and scaling MCP services.

MCP Just Handed You a Deprecation Policy. Steal It.

A practical framework for applying MCP’s new feature lifecycle to internal tools and connectors, covering ownership, versioning, migration paths, sunset dates, and communication with dependent teams.

MCP Graduates to Enterprise Infrastructure

A broad technical overview of MCP’s stateless architecture, authorization hardening, formal feature lifecycle, and official extensions for interactive interfaces and durable long-running operations.

More MCP on August 6

Couldn’t make one of the launch parties, or still working through what 7-28 means for your systems?

MCP Jam is a free virtual event on August 6, featuring technical talks, live demos, and lessons from developers, maintainers, and platform teams building with MCP.

Register here

JOB OF THE WEEK

Principal ML Platform Engineer // Synthesia // Europe (Remote)

Synthesia develops AI video software for business use. This principal-level IC role shapes the ML platform supporting generative model training, evaluation, deployment, and serving, with responsibility for infrastructure reliability, automation, GPU workloads, internal tooling, and architectural direction.

Responsibilities

• Design platform systems for model training, evaluation, and production serving.

• Build reliable, scalable infrastructure and tooling for ML workloads.

• Improve GPU workload scheduling, monitoring, debugging, and resource efficiency.

• Translate researcher and product needs into robust platform capabilities.

Requirements

• Build and operate reliable, scalable, maintainable production systems at scale.

• Work hands-on with cloud infrastructure, Linux, and infrastructure automation.

• Operate Kubernetes-based distributed workloads in complex production environments.

• Write strong Python code for backend systems and internal tooling.

  ---

MLOPS COMMUNITY

What an Anthropic Engineer Thinks About MCP

MCP’s biggest protocol change removes persistent server sessions, clearing a major scaling obstacle for deployments handling millions of requests.

• Stateless transport lets load balancers move requests between servers without synchronizing session data.

• Multi-round-trip requests carry required context across tool calls while preserving agent state.

• SDK updates should keep migrations small, while tool search and programmatic calling reduce context bloat.

The release shifts MCP closer to standard web infrastructure without stripping away the state agents still need.

Video · Spotify · Apple

AI Hype vs. Real Value

AI projects can look productive until token and compute bills erase the gains. The discussion argues that real value comes from redesigning complete workflows, rather than scattering pilots across the business.

• Compare AI-assisted teams with control groups and measure changes in speed, cost, revenue, or output.

• Match models to tasks, limit token use, and apply FinOps at workflow level.

• Keep humans involved in sensitive decisions, then reduce oversight as accuracy improves.

AI pays off when workflow gains survive cost and risk checks.

Video · Spotify · Apple

How iFood Designed Rosie Around Latency

When dinner is late, a fast wrong answer can cost more than a slow correct one. iFood’s support agent is built around that production reality, treating latency as something to design for.

• Async callbacks keep slow tools from blocking the conversation.

• Rollback merges rapid-fire messages before the agent replies.

• A small reviser model checks outgoing messages, while end-state evals measure whether the user’s problem was resolved.

The system stays responsive without assuming every dependency will be fast.

Read the blog

IN-PERSON EVENTS

• San Francisco - July 30

• Pune, India - August 1

• New York - August 14

• Toronto - September 10

• San Francisco, Voice Agents Forum - September 16

VIRTUAL EVENTS

• MCP Jam - August 6

  ---

MEME OF THE WEEK

ML CONFESSIONS

Buying Time

The support tickets all said the same thing. Requests hanging for twenty seconds, sometimes longer. I would open the dashboard, see p95 sitting under 500ms, and close it. The graph and the tickets could not both be right, and I decided it was the tickets.

I raised the client timeout. A week later I raised it again. I told the team it was almost certainly upstream, and I believed it, because the number I looked at every morning kept coming back clean.

The number came back clean because we wrote it only after inference returned. Every request that timed out or lost its worker died before it reached that line. I had a dashboard that could report on the requests that worked and nothing else.

We added a start event and a timeout counter. The next incident showed up on the graph while it was still happening.

Share your confession here.

Share

Hit reply with the buzzword you’re hearing most. As well as joining the leaderboard it’ll also help make sure the newsletter reaches you when it starts coming from demetrios@aaif.live as part of our move to the Agentic AI Foundation.

We might even use them to make a bingo card like this week’s meme.

HOT TAKE

Who You Gonna Call?

Models pick the wrong tool long before a server hits 1,000 of them.

Who should own tool overload, server or client?

SERVER or CLIENT

LAST WEEK’S TAKE

The Next Re-Generation

Agent-built internal tools now have the life expectancy of a Star Trek redshirt.

EXTRA 15% OFF EARLY BIRD

Meet the People Building MCP

AGNTCon + MCPCon brings together the people building MCP and agent systems in production, with technical sessions covering architecture, infrastructure, interoperability, security, observability, and more.

Use promo code COMMUNITY15 for an extra 15% off the early-bird price.

North America: San Jose, October 22–23
Early bird ends July 28

Europe: Amsterdam, September 17–18
Early bird ends August 2

HIDDEN GEMS

Curated finds to help you stay ahead

Coding Agent Benchmark on Real PRs

Built from recent, human-written PRs across a multi-million-line codebase, the benchmark compares model and harness quality, task-level cost, context efficiency, and performance across different coding complexities.

Official MCP Certification Exam Scope

Covering architecture, message flow, tool invocation, security, governance, and ecosystem use cases, the vendor-neutral MCPA sets out the foundational MCP knowledge developers need for the exam.

Agent Infrastructure Trends Report

Drawing on 30 days of AI Tinkerers demos, the report tracks emerging patterns in cognitive architecture, context-cost optimization, and agent harnesses for reliability, governance, security, and production deployment.

One-Click Hermes Agent Deployment

Combining one-click Docker deployment with persistent memory, multi-platform messaging, scheduled tasks, subagents, and support for 200+ models, the service offers a managed VPS route to running Hermes Agent.

JOB OF THE WEEK

Senior Data Engineer // WalletConnect // Remote, UK

WalletConnect develops infrastructure connecting wallets and applications across blockchain networks. The Senior Data Engineer will build and operate event-driven pipelines, real-time processing systems, and backend data services supporting payments, financial reconciliation, merchant reporting, operational monitoring, and product analytics.

Responsibilities

• Build and operate event-driven pipelines for high-volume payment transactions.

• Scale real-time data systems from MVP to production architecture.

• Model ledger data for reconciliation, settlements, fees, and reporting.

• Develop backend services exposing data through internal platforms and APIs.

Requirements

• Five-plus years building and operating production data systems.

• Expertise in SQL, Python, and large-scale data processing.

• Experience with Airflow, dbt, ClickHouse, and event-driven architectures.

• Comfortable reviewing and productionizing code from AI-assisted development tools.

  ---

MLOPS COMMUNITY

What Happens When Every Developer Has 20 AI Agents

AI can generate code faster than teams can review, secure, deploy, or even remember where they put it. The result may be a flood of disposable apps and agents rather than a clean productivity gain.

• Cheap code shifts the burden toward maintenance, governance, and deciding what deserves to survive.

• Build systems, package repositories, security reviews, and maintainers are already absorbing more traffic and more work.

• MCP adoption is rising quickly because it connects capable models to useful internal data.

The advantage will go to teams that can turn cheap code into software they can still trust and operate.

Video · Spotify · Apple

The Creator of FastMCP Explains the Future of MCP

Give an agent raw access to a data warehouse and it may answer one question by taking the warehouse down and running up the bill. MCP’s strongest use may be controlled internal access rather than public-facing integrations.

• FastMCP turns Python functions into servers while handling schemas and protocol details.

• Curated tools can make company data self-serve without exposing unrestricted queries.

• MCP Apps return interactive tables and charts without stuffing every row into the LLM context.

The useful layer is the guardrail between agents and the systems they touch.

Video · Spotify · Apple

A Discussion on Loop Engineering

A coding loop can launch 75 agents and burn through its token budget in minutes if its controls are wrong. Making agents autonomous requires more than repeating prompts until the tests pass.

• Machine-readable plans, tests, and independent reviewers give each loop a measurable target.

• Observability and evals show where systems drift, stall, or waste tokens.

• Persisted failure modes and learnings let later runs avoid repeating the same mistakes.

The useful loop is the one that knows when to stop.

Watch the reading group

Agent Docs: Answer Locally Before the Web — A Shared Reference for Every Coding Agent

Your coding agent can be perfectly faithful to the wrong documentation. Stale search results and repeated web lookups add latency, token cost, and version-specific errors that often surface only when the code breaks.

• A local .agents/docs/ map points agents to approved upstream pages before they search.

• Project-scoped references stay diffable in Git and consistent across coding tools.

• Verification dates, changelogs, and stale warnings make outdated guidance visible.

The result is faster retrieval with fewer wrong-version mistakes.

Read the blog

MCP RELEASE PARTIES

• Amsterdam - July 29

• London - July 28

• New York - July 28

• San Francisco - July 28

• Seattle - July 28

IN-PERSON EVENTS

• San Francisco - July 30

• Pune, India - August 1

• San Francisco, Voice Agents Forum - September 16

VIRTUAL EVENTS

• Coding Agents Lunch & Learn Session - July 24

MEME OF THE WEEK

ML CONFESSIONS

Green Every Day

I set up a drift check for our fraud model that compared each day’s scores with the previous day. It stayed green for nearly three weeks.

The average score had fallen from 0.41 to 0.18 over that period, slowly enough that no single day crossed the threshold. I was checking the merchant category report most mornings too, working down the merchant list from the top and stopping once the names stopped being ones I recognized.

The manual review queue was down by about half before someone in finance asked whether we had changed the fraud policy.

We had not. A vendor had updated its merchant category taxonomy, and a lot of smaller merchants were now landing in “unknown.” Unknown was rare in training and mostly legitimate when it did appear, so the scores came down.

Everything that moved was further down the list than I ever got.

Share your confession here.

Share

HOT TAKE

Commitment Issues

Most agent-built internal tools should be disposable by default.

Maintain or Regenerate?

MAINTAIN or REGENERATE

LAST WEEK’S TAKE

No Reminder Needed

We’ll remember not to ask twice when it’s this clear you’re choosing memory over long context for agents.

HIDDEN GEMS

Curated finds to help you stay ahead

Persistent Memory for Coding Agents

Local memory for AI coding agents, with MCP, REST, hooks, search, and project context that can persist across sessions and tools.

Codebase Graph Memory for Agents

Persistent codebase knowledge graphs, served through MCP, give coding agents structural search across functions, classes, call chains, routes, and cross-service links without repeated file-by-file exploration.

Distributed AI Inference Deployment Guide

Six traffic-shape blueprints pair vLLM and llm-d topologies with TTFT/TPOT diagnostics, mapping how teams can move from single-instance serving to distributed inference.

1,000-Token Agent Inference Engine

Running Kimi K2.7 Code at model-native precision on a single 8×B200 node, Lithos reports inference above 1,000 tokens per second for faster coding-agent workflows.

Agentic AI Momentum Report

What 116 Agentic AI Projects Reveal

The open-source agent stack is attracting contributors and adoption quickly, but popularity does not show whether a project is secure, responsive, or structurally resilient. AAIF’s new Momentum Report compares 116 projects across five layers using adoption, development health, maintainer activity, vulnerabilities, governance, and contributor concentration.

Frameworks and infrastructure lead adoption with 1.5 million GitHub stars and 226,000 forks. Agent applications tell a different story: they attract the most active contributors and resolve issues in roughly two weeks, with pull requests merged in about two days.

Security presents a less encouraging picture. Unpatched CVEs across the projects rose 2.6x between December 2025 and June 2026, reaching around 9,200. Framework and infrastructure projects account for nearly half of them.

The governance data also separates formal processes from genuine contributor diversity. Some projects meet OpenSSF governance checks but remain controlled by one organization, while others have broader participation but weaker documentation.

Read the report and compare individual projects.

JOB OF THE WEEK

Member of Technical Staff - Platform // Runlayer // Hybrid NYC / Remote (US Timezones)

Runlayer is hiring a platform engineer to build the systems behind enterprise MCP, agent, and workflow deployments. The role covers scalable platform architecture, integrations, data transformation, customer requirements, and rapid feature iteration in production AI environments.

Responsibilities:

• Architect scalable systems for customer MCP, agent, and workflow deployments.

• Build APIs, integrations, and data transformation workflows for enterprise systems.

• Translate customer requirements into platform features and technical delivery plans.

• Prototype, validate, and ship features based on real usage feedback.

Requirements:

• 7+ years building and operating production software systems.

• Experience designing APIs, databases, integrations, and enterprise platform services.

• Background working with AI/ML systems or related infrastructure.

• Strong communication across engineering, product, and customer-facing technical teams.

MLOPS COMMUNITY

Developers May Stop Depending on Libraries

Models can now run 80 to 100 tool-call loops, while one compact tool can expose a huge capability surface without flooding the context window.

• RL has improved how agents chain actions, navigate environments, and use minimal tool interfaces.

• Small MCP surfaces can generate code, query large hubs, and return results without replaying everything through the model.

• MCP apps can place rich results directly in the UI, reducing token waste.

The design question is shifting from tool count to how efficiently each tool expands what an agent can do.

Video · Spotify · Apple

10 Cities. 4 Countries. One Unexpected MCP Lesson.

An MCP server can take seven lines of Python, but each exposed tool consumes context and creates another surface to secure.

• MCP does not make an API intelligent. It gives an LLM a structured way to select and call functions.

• Strong designs expose a small set of common actions, with clear annotations, validation, authorization, and limits on costly requests.

The practical lesson is to treat MCP as a controlled interface for model-driven API use, not a layer of magic.

Video · Spotify · Apple

Best Model Serving Runtimes To Build Optimized ML APIs

A Flask container can become an expensive bottleneck once inference traffic rises and latency starts to matter. This guide compares four production serving options.

• TensorFlow Serving and TorchServe suit teams committed to one framework.

• BentoML favors Python flexibility, broader compatibility, and simpler implementation.

• Triton offers stronger GPU utilization and optimization, with considerably more operational complexity.

The right choice depends on whether your priority is development speed, framework support, or squeezing more predictions from each machine.

Read the blog

IN-PERSON EVENTS

• Munich - July 23

• San Francisco, Voice Agents Forum - September 16

VIRTUAL EVENTS

• Coding Agents Lunch & Learn Session - July 17

• A Discussion on Loop Engineering - July 21

MEME OF THE WEEK

ML CONFESSIONS

Speak for Yourself

We were testing a voice agent for booking appointments, and it kept interrupting people before they’d finished giving their ZIP code. I adjusted the endpointing settings until it stopped happening and the test calls looked much better.

A few weeks later, support started getting complaints that the agent was taking ages to respond. Someone on the team said, “It’s going full mute for five seconds after every sentence.” It turned out I’d mostly tested it myself, and I speak quickly with short pauses. The new setting worked fine for me, but for anyone who paused naturally between parts of an answer, the agent sat there waiting to see if they’d carry on.

We went through recordings and tuned it against a much wider mix of speakers.

Share your confession here.

Share

HOT TAKE

Out the Window

A bigger context window is not the same as memory. It is just a more expensive place to forget things.

What matters more for agents: long context or persistent memory?

CONTEXT or MEMORY

LAST WEEK’S TAKE

Access Granted

Have to trust the accuracy of the people who accessed the poll to tell us what matters more in production.

HIDDEN GEMS

Curated finds to help you stay ahead

Real-Money Agent Guardrail Patterns

Model judgment is separated from deterministic validation, with five production patterns for keeping mortgage-servicing agents from committing unsafe actions across arithmetic, permissions, tool use, and recovery.

CLI Coding Agent Market Report

Terminal-first agent use has matured into a crowded 35-tool field, with practical comparisons across lab CLIs, platform products, open-source harnesses, memory, orchestration, cost, and governance for choosing where to standardize.

Codebase Audit Planning Skill

Senior-model review is turned into self-contained implementation plans, with strict read-only rules, evidence requirements, executor handoff steps, and review gates for cheaper coding agents.

Team Prompt Library Tool

Shared skills package colleagues’ strongest prompts into a searchable, versioned library, giving teams reusable AI workflows with explicit access controls and an open-source SKILL.md toolchain.

JOB OF THE WEEK

Senior Engineer, Platform & Data // Mosaic Pediatric Therapy // Remote / Charlotte, NC

Mosaic Pediatric Therapy is hiring a senior platform and data engineer to support internal applications, AI-assisted operational tools, DevOps practices, and Azure Fabric Lakehouse pipelines across clinical, billing, credentialing, and analytics workflows involving sensitive healthcare data.

Responsibilities:

• Establish standards for internal applications and AI-enabled operational tools.

• Improve monitoring, alerting, incident response, and deployment workflows.

• Maintain Azure Fabric Lakehouse pipelines across bronze, silver, gold layers.

• Support reporting, tooling, and self-service data access for stakeholders.

Requirements:

• 4+ years supporting production software, data platforms, or cloud infrastructure.

• Strong SQL, Python, Git, testing, debugging, and code review skills.

• Experience building and maintaining production ETL or ELT pipelines.

• Familiarity with Claude Code, Copilot, Cursor, or similar workflows.

MLOPS COMMUNITY

The Next Programming Language Is English

Natural language is becoming a programming interface, but it does not come with the guarantees developers are used to getting from formal languages.

• Why AI coding changes the abstraction stack, replacing deterministic translation with probabilistic systems that need new safety nets.

• How durable execution gives agents and tools a logical process that can survive client failures, network drops, and infrastructure chaos.

• Why MCP tasks matter for long-running work, from streaming analysis to invoice processing, human approvals, durable timers, and scale-to-zero infrastructure.

The big shift is a programming model that keeps agent work alive when prompts, networks, tools, and humans fail.

Video · Spotify · Apple

AI Agents Should Be Treated Like Hackers

Your agent does not need every API. It needs the right slice, at the right time, with proof that it should have it.

• MCP can connect agents to enterprise systems, but most APIs were built for trusted engineers, not autonomous software.

• GraphQL offers a useful pattern for agent access because schemas expose business meaning and support field-level permissions.

• The next challenge is richer context, better access patterns, audit logs, and accountability around business decisions.

Faster agents need narrower doors and better receipts.

Video · Spotify · Apple

Omnigent: Composition, Control, and Collaboration for AI Agents

Your agent may be right, but it can still burn money on the wrong model.

• Why agent workflows need portable context, memory, and configuration across models and harnesses.

• How forking conversations and agent debates can expose missed questions before work goes sideways.

• Why tokenomics needs both developer-level visibility and centralized governance.

The payoff is practical control over agent work without pretending one model fits every task.

Video · Spotify · Apple

IN PERSON EVENTS

• Munich - July 23

• San Francisco, Voice Agents Forum - September 16

VIRTUAL EVENTS

• Coding Agents Lunch & Learn Session - July 10

• A Discussion on Loop Engineering - July 16

MEME OF THE WEEK

ML CONFESSIONS

Overconfidence Interval

We shipped a recommendations model once and everyone thought it had gone fine.

The numbers looked a bit better, nobody complained, and the person who’d led the build was being quietly smug about it, which was fair enough because it had been a horrible project.

About a week later, someone on the team messaged me saying one region looked “a bit too boring.” Same traffic pattern every day, barely any movement, almost like nothing had changed.

I checked the config and realized that region was still pinned to the old model because one of the rollout flags had been copied from staging and never updated.

So about a third of the traffic was not actually using the model we thought we’d shipped.

I sent it to the person who’d done the deploy, trying very hard not to sound pleased about it, because he had spent the previous few days telling everyone how clean the rollout had been.

To be fair, he fixed it quickly. But then we had to go back through the results and work out which numbers were real, which were old-model traffic, and which were basically just us congratulating ourselves on a config mistake.

Share your confession here.

Share

HOT TAKE

Access All Errors

The scariest agent isn’t the one that hallucinates. It’s the one with the right answer and the wrong permissions.

What matters more in production: accuracy or access?

ACCURACY or ACCESS

LAST WEEK’S TAKE

A Relevant Take

A not-irrelevant number of people still chose relevance when retrieval gets sensitive.

HIDDEN GEMS

Curated finds to help you stay ahead

Safer Coding Agent Loops

Loobster gives Claude Code and Codex a repeatable plan-build-test-secure loop with approval gates between phases, so agent work is easier to review before merging.

Internal AI Engineering Stack Guide

Built around Cloudflare’s R&D workflow, the stack combines AI Gateway, Workers AI, MCP portals, AGENTS.md files, and automated code review for scaling agentic engineering work.

Model Serving Community Report

Across vLLM, KServe, llm-d, Kubernetes inference work, and OGX, recent releases point to harder production questions around routing, observability, hardware support, and agentic workloads.

Speculative Decoding Codebase

Training and evaluation utilities cover data preparation, draft-model training, checkpoints, and benchmark testing for comparing speculative-decoding methods across Qwen, Gemma, Eagle3, DFlash, and DSpark.

JOB OF THE WEEK

Software Engineer – Backend // Moss // San Francisco, CA, US / Remote

Moss builds real-time semantic search infrastructure for conversational AI. This backend role focuses on production retrieval systems, including APIs, caching, indexing, observability, cloud and edge infrastructure, and reliability work for latency-sensitive customer deployments at scale.

Responsibilities

• Build query path, APIs, caching, indexing, and data plane.

• Improve latency, correctness, and reliability across live retrieval workloads.

• Add observability, load testing, alerting, and incident response processes.

• Debug customer deployments and convert failures into infrastructure improvements.

Requirements

• 5+ years operating latency-sensitive production backend systems.

• Strong distributed systems fundamentals across APIs, databases, caches, queues.

• Experience with monitoring, incident response, capacity planning, and SLAs.

• Familiarity with regulated environments, ideally SOC 2 or HIPAA.

  ---

MLOPS COMMUNITY

Coding Agents Are Secretly General Agents

The scary bit isn’t that agents can write code. It’s that code lets them build their own tools.

• Code gives agents a tight feedback loop: compile, run tests, measure performance, pass failures back, and improve.

• That makes coding agents a route into broader knowledge work, where sandboxes and scripts can reach business systems.

• The harder problem becomes context: tasks, docs, chats, decisions, permissions, and where agents are allowed to act.

The agent stack starts to look less like a chatbot and more like an operating layer for work.

Video · Spotify · Apple

Can AI Agents Be Trusted in Healthcare?

A patient gets 25 pages of discharge instructions. An agent can help, but in healthcare, the dangerous question is what it is allowed to do.

• MCP sits above the data layer, enforcing role-specific, time-bound access for patients, providers, and caregivers.

• Clinical and regulatory steps stay deterministic, with providers approving decisions before action.

• Restricted tools, trace logs, and behavior metadata help catch drift and prove auditability.

In regulated AI, permissioning becomes the product.

Video · Spotify · Apple

Building the Star Trek Future: Why I Swapped My Keyboard for a Council of Agents

The keyboard isn’t disappearing; the job is moving up the stack. This piece looks at the shift from writing every line to orchestrating agents, with trust, oversight, and shared memory as the real constraints.

• Agentic coding changes the role from implementation to intent, architecture, review, and testing.

• Speed still needs vigilance, especially around security, UX, documentation, and production readiness.

• The missing layer is persistent context across agents, tools, and decisions.

The hard part is no longer only generating work, but keeping agents grounded, connected, and accountable.

Read the blog

The Current State of Agentic Retrieval - Qdrant Roundtable

Agents don’t just need more context. They need to stop rummaging through it like a junior analyst with infinite API credits.

• Retrieval quality can be checked cheaply with score spreads and statistical signals before paying for reranking or another LLM call.

• Search skills can teach agents when to use dense, sparse, lexical, or hybrid retrieval.

• Long-term memory needs forgetting, deduplication, and graph/vector structure, not just bigger context windows.

Agentic retrieval is becoming less about finding chunks and more about controlling search behavior.

Watch the roundtable

IN PERSON EVENTS

• Munich - July 23

• San Francisco, Voice Agents Forum - September 16

VIRTUAL EVENTS

• Coding Agents Lunch & Learn Session - July 3

  ---

MEME OF THE WEEK

ML CONFESSIONS

Changelog

v0.8.0 Added a model registry.
v0.9.0 Changed the scorer to pull the model tagged latest.
v1.0.0 Shipped a retrained ranker. Offline metrics looked good.
v1.0.1 Found out latest meant the model artifact, not the feature definitions.
v1.0.2 Production was serving the new model with the old transforms. Half the numeric features scaled wrong. One categorical was mostly unknowns.
v1.0.3 Rolled back. Rankings went up.
v1.0.4 Added feature-set versioning. Added a deploy check.
v1.0.5 Pinned the model. Pinned the feature set. Made releases slightly slower.

Share your confession here.

Share

‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏

Positive news from OpenAI on open-source security, though the name sounded familiar...

HOT TAKE

RAGrets

Your RAG eval can be green while the wrong user sees the right chunk.

When retrieval gets sensitive, what matters more - relevance or authorization?

RELEVANCE or AUTHORIZATION

LAST WEEK’S TAKE

On the Trail

Evals are trailing, with 80% putting more trust in audit logs when agents fail.

HIDDEN GEMS

Curated finds to help you stay ahead

Severe Code Review Checklist

Cursor’s strict code review checklist targets maintainability, structural simplification, oversized files, messy branching, weak abstractions, and type-boundary problems that can block approval.

CLI Sandbox for AI Agents

For running agents and risky CLI commands, Fence applies filesystem rules, default-deny network access, command blocks, and reviewable policies on macOS and Linux without requiring a VM or container.

TypeScript Harness for Autonomous Agents

For building autonomous agents, Flue provides a TypeScript harness with sessions, tools, skills, filesystem access, sandboxed execution, workflows, telemetry, and local or hosted deployment options.

Cloudflare’s Agent Engineering Stack

Inside Cloudflare’s internal rollout, engineers connected AI Gateway, Workers AI, MCP portals, AGENTS.md, Backstage, and AI code review to support secure coding-agent use at scale.

OPEN SOURCE SOFTWARE

Omnigent, a control layer for messy agent workflows multiple agents

Agent work is starting to look a lot like browser-tab chaos for terminals. Claude Code in one terminal, Codex in another, maybe Pi or a custom agent for a specific job, a sandbox or two, a pile of credentials, and an approval step that lives in someone’s head. Work gets copy-pasted between agents, Docs, and Slack. Each harness only understands its own sessions, so nothing carries over when you switch tools.

Earlier this month, Databricks open sourced Omnigent under Apache-2.0. It calls itself a meta-harness: a common interface that wraps Claude Code, Codex, Pi, and agents you write yourself, then adds the things a single harness rarely gives you across tools, like collaboration, policies, cost caps, and cloud execution.

What it is

The core idea is that, however each harness talks to its model internally, the interface to the user is the same: messages and files in, text streams and tool calls out.

Omnigent builds one API over that and splits the system in two. A runner wraps an agent in a sandboxed session and executes the loop and tools, either on your machine or in a cloud sandbox. A server handles persistence, policies, sharing, and access from terminal, browser, phone, or the macOS app.

The split keeps code execution and model credentials on the host side rather than in the server, which matters if you care where your secrets live. (cough cough enterprise-ready)

Omnigent wraps agent harnesses in a runner, with a server handling policies, history, MCPs, skills, and shared interfaces. Image: Databricks

Why it matters

Let’s look at what this lets you build once the agents are interchangeable:

• Collaboration on the session, not the chat window. Share a live session by URL, watch it run, comment on files, co-drive it, or fork it. The session is the shared object, so a teammate or reviewer joins the work rather than a transcript of it.

• Contextual policies. Rules that track session state instead of static allow/deny. The example from the docs is gating git push only after an agent has installed a new npm package. Policies stack across server, agent, and session, with the stricter rules checked first.

• Cost caps. Per-session spend tracking with hard caps and soft warnings, so a sub-agent tree cannot quietly burn through tokens. You can tell it to pause and ask after every $100. (Or if you are like me, have it play a cha-ching sound and Fat Joe/Lil Wayne make it rain at full volume)

• Cloud execution. This is becoming table stakes now. Nobody is excited about keeping their laptop open on the metro.

• Multi-harness authoring. Probably the most interesting for me. Define an agent in one YAML file covering prompt, executor, tools, policies, and sub-agents, then switch harnesses with a one-line change, or mix harnesses inside the same agent.

Meta harness & Loop engineering

Addy Osmani’s recent writing on loop engineering argues that the work has shifted from prompting one agent to designing repeatable systems of worktrees, sub-agents, connectors, and persistent state.

Omnigent reads as a control layer for that shift. The bundled Polly example is a lead agent that plans, hands coding to sub-agents in parallel git worktrees, and routes each diff to a reviewer from a different vendor than the one that wrote it. Debby runs the same question through a Claude head and a GPT head side by side and lets them debate.

So the fit is workflows like planner plus coder plus reviewer, cross-vendor adjudication, and a team working a single repo state together. The current docs lean harder on orchestration, collaboration, and policies than on scheduling and automation, so today Omnigent is less a finished loop product and more a substrate you build loop-style workflows on.

What looks promising

• A real abstraction if you already run more than one harness or model family, rather than a wrapper around one.

• A more explicit governance story than a lot of open-source agent tooling: contextual policies, spend caps, and approval gates built in.

• Collaboration primitives that are part of the session model, not bolted on afterward.

• A YAML agent spec that makes multi-agent, multi-harness setups legible for a platform team.

What needs caution

This is alpha. The repo is clearly early, carries an explicit alpha badge, and there is not much independent operator feedback yet, so almost all the public evidence is launch material. Treat any judgment on ergonomics or reliability as provisional until people have run it on real repos.

The setup surface is also wider than the one-line install suggests. You need Python 3.12+, uv, git, and Node.js 22+ for the Claude/Codex/Pi harnesses, plus tmux for the native wrappers. Remote deploys add database choices, memory floors, slower first boot against remote Postgres, and provider caveats like Modal’s 24-hour sandbox lifetime and Daytona’s free-tier egress limits.

The biggest operational risk is that the collaboration features cut both ways. Co-drive means another user’s messages can execute on your machine, and the docs warn that header-based auth is dangerous unless a trusted reverse proxy rewrites identity headers correctly. Get that wrong and one caller could impersonate another and reach sessions, files, and history. Cross-vendor loops can also raise token spend fast, and if your team is standardizing on one harness anyway, Omnigent may be more control plane than you need.

In practice

I got early impressions from my friend Mateo (who is a much better engineer than myself).

Setup was genuinely easy, which is worth noting against the dependency list above. His take was that Omnigent simplifies the loopcraft idea a lot, and that for someone already running their own custom multi-agent setups the tradeoff is versatility against ease.

Bespoke stacks still offer more flexibility, and can go further than Omnigent’s static YAML-led setup. Gas Town and the newer Gas City SDK, for instance, can run a coordinator agent that watches a task tracker and hands work out to other agents on its own. But, the Gas projects are also far harder to stand up, and Omnigent closes most of that gap with much less effort.

The main limitation was around MCP. Today, an MCP server is wired in either as a local command or a remote URL with a bearer token, which means keeping tokens on your machine or in the Databricks config. He would prefer MCP OAuth with dynamic client registration so credentials are not sitting locally. I’m sure this will change soon. (Friend of the pod Corey Zumar is doing a ton of work on Omniagent, and that guy ships)

Between CLIs, MCP, and Agent Skills, switching between harnesses is already possible without much ceremony, and a team running one or two agents may find their own light glue does the job.

I know in our Friday coding agent sessions folks have asked how others are using coding agents in teams now. The case for Omnigent gets stronger as the coordination problem grows: several harnesses, multiple people, work moving across machines, and policies or spend limits that need to hold across all of it.

JOB OF THE WEEK

Senior AI Software Engineer, Internal Enablement // Extend // Remote US

Extend is hiring a senior engineer to build internal AI tooling for non-engineering teams. The role focuses on secure MCP connectors, reusable agent skills, credential management, observability, and infrastructure for turning prototypes into reliable internal tools.

Responsibilities:

• Design secure MCP connectors across finance, CRM, data, support systems.

• Build shared agent skills and review patterns for internal teams.

• Build connector agents that scaffold MCP servers and provision access.

• Instrument MCP calls, skill execution, and credential use with OpenTelemetry.

Requirements:

• Five-plus years shipping production systems and improving failure points.

• Strong Python, TypeScript/Node, or Rust production engineering experience.

• Experience building OAuth integrations, webhooks, retries, and rate limits.

• Familiarity with AWS, IAM, KMS, Lambda, and serverless risks.

MLOPS COMMUNITY

The Dark Side of MCP Servers

A broken MCP server can make a capable agent look useless, and a malicious one can change the tools it exposes after you already trusted it. This podcast looks at what has to improve before MCP feels safe in production.

• Toolbench and MCP Debugger expose protocol failures, misstated capabilities, broken auth, and bad tool schemas.

• Skills, CLIs, and MCP solve different problems once agents move beyond local coding workflows.

• OAuth-based MCP keeps user identity outside the prompt, reducing spoofing and token misuse.

Reliable agents need boring protocol behavior before they get dangerous work.

Video · Spotify · Apple

Sandboxing, Agent Harnesses, and Agent Teamwork

An SRE agent that gets 70% right can still send an engineer down the wrong path during an incident. This podcast looks at why production agents need more than better models.

• Harness design has to evolve as models improve, relaxing rigid tools while adding stronger sandboxing.

• Verification matters more than the agent loop, especially across messy production environments.

• Slack history, code, logs, and human feedback can teach agents local rules and decision bounds.

Better agents need clear boundaries, fast feedback, and humans reviewing the decisions that matter.

Video · Spotify · Apple

Roundtable: Building Real-Time ML Systems with Zipline + Chronon

A feature can look right offline and still be wrong when it hits serving. This round table looks at why teams are moving feature pipelines toward Chronon and Zipline as models, ranking systems, and real-time use cases grow.

• Batch and streaming definitions can share one authoring path, reducing training-serving drift.

• Point-in-time joins, backfills, and online lookup become platform concerns instead of product-team chores.

• Adoption still carries cost, especially around debugging, versioning, and fitting existing data assumptions.

For growing ML systems, feature correctness is where speed either holds or breaks.

Video · Spotify · Apple

Securing the Agent: Vendor-Neutral, Multitenant Enterprise Retrieval and Tool Use

A RAG system can retrieve the perfect chunk and still show it to the wrong person. This discussion looks at the security gap behind multi-tenant agent systems, where relevance scores alone are not enough.

• Authorization has to happen at chunk level, before retrieved context reaches the model.

• Predicate pushdown helps preserve recall by filtering tenant access inside the vector database.

• Server-side orchestration gives teams a stronger place to enforce policy on tool calls and retrieval.

The danger starts when retrieval works, but access control arrives too late.

Watch the reading group

Supagents: One Source to Rule Your Coding Subagents

One Friday tweak to an agent persona can quietly turn into five different behaviors across five coding tools. This blog shows how supagents keeps Claude Code, Gemini CLI, Copilot, Cursor, OpenCode, and Kilo Code aligned from one markdown source.

• Shared instructions live in .agents/supagents/, then compile into each tool’s expected agent format.

• Per-tool frontmatter stays verbatim, so fast-changing agent config does not need a fragile translation layer.

• Pre-commit and CI checks stop generated agent files drifting from the source.

A small compiler can save a lot of debugging when every agent starts from the same instructions.

Read the blog

VIRTUAL EVENTS

• The Current State of Agentic Retrieval - June 25

• Coding Agents Lunch & Learn Session - June 26

  ---

MEME OF THE WEEK

ML CONFESSIONS

Feature Drift Store

We added a feature store so the training pipeline and the serving path would read the same features. The whole point was to stop the two drifting apart.

A churn model had been live for about five months with no obvious issues. Then retention asked why a batch of flagged accounts had canceled weeks earlier. We spent two days assuming it was a labeling problem on their end.

It wasn’t. The serving path was reading the feature store. The training pipeline was still reading the old offline tables nobody had pointed at the store. Same feature names, same schema, values computed a day apart. Close enough that nothing looked broken and far enough off to matter.

I didn’t have a clean fix. I backfilled the store from the offline tables so at least they matched, and left the proper migration as a ticket. It’s still open.

Share your confession here.

Share

‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏ ‌​‍‎‏

Shop // View Online

June 18, 2026

Forwarded this email? Subscribe here

Nice to see a small startup take off.

HOT TAKE

Trail Mix

The next agent outage won’t be a hallucination. It’ll be a tool call nobody logged.

What do you trust more - evals or audit trails?

EVALS or TRAILS

LAST WEEK’S TAKE

Talk therapy

It was close, but chat edged it - most would rather talk through their debugging than shout at their dashboard.

PRESENTED BY QDRANT AND MLOPS COMMUNITY

The Current State of Agentic Retrieval

Join MLOps Community and Qdrant on June 25 for a virtual roundtable on how agents retrieve and use context.

The discussion will cover vector search applications for agents, including agent tools, embedding methods, agentic evals, memory, and the challenges still shaping how retrieval works in agentic systems.

Featuring Ewa Szyszka, Dylan Couzon, Neil Kanungo, and Evgeniya Sukhodolskaya from Qdrant.

Register here

HIDDEN GEMS

Curated finds to help you stay ahead

A study of agent safety under real-world pressure, tracking how tool-using agents handled social engineering, memory, email, shell access, and cross-agent safety risks.

Exploring whether newer models feel more argumentative and are trading conversational usefulness for guardrails, anti-sycophancy, and coding performance.

Using Agent Substrate for Kubernetes agents, kagent can run AI agents in warm worker pools with fast suspend/resume, snapshotting, sandboxing, and policy-controlled network traffic.

A portable format for sharing agent-readable context, letting teams package metadata, runbooks, schemas, metrics, and documentation as markdown files that agents and humans can read across tools.

JOB OF THE WEEK

Senior Data Engineer // DataSF, Office of City Administrator // San Francisco, CA / Hybrid

DataSF, part of San Francisco’s Office of City Administrator, is hiring a Senior Data Engineer to build and maintain city data infrastructure, including Snowflake administration, data pipelines, IaC, monitoring, and support for analytical and operational use cases.

Responsibilities

• Administer Snowflake access, security, monitoring, performance, and cost controls.

• Build resilient batch and streaming pipelines from varied data sources.

• Use Terraform to manage reproducible, version-controlled data infrastructure.

• Monitor platform health, troubleshoot issues, and maintain data quality.

Requirements

• Three years’ relevant experience maintaining enterprise infrastructure components.

• Experience with Snowflake, BigQuery, Databricks, or similar platforms.

• Strong SQL, Python, dbt, and data modeling experience.

• Knowledge of data governance, privacy, security, and ELT principles.

  ---

MLOPS COMMUNITY

From Single-Player to Multi-Player: Operating AI Agents at Scale

One engineer burned through an entire Anthropic budget in seven hours from a laptop. That is the problem here: agents are useful, but they need infrastructure built for non-deterministic actors.

• Policy needs to cover access, actions, budgets, sandboxing, compliance, and eval-based confidence.

• Agent workloads strain old assumptions around source control, search, runtime, audit logs, and provenance.

• Composable agents may matter more than one giant agent, especially as shared context becomes critical.

The risk is treating agents like scripts when they behave more like unpredictable teammates.

Video · Spotify · Apple

Agents & the $40M Bet on Multiplayer AI

When an agent starts a week-long task, one person cannot steer it alone. This discussion looks at the shift from solo AI sessions to shared workspaces where agents and humans hand work across teams.

• Longer agent tasks need shared state, hosted sessions, and clearer handoffs between specialists.

• Product design may matter as much as model quality for real collaboration.

• Fast-changing models make planning, pricing, and alignment harder.

The next AI bottleneck may be coordination, not capability.

Video · Spotify · Apple

MCP Servers Are Becoming the UI for AI Agents

Your MCP server can fail only for Cursor users, burn context with giant responses, or leave agents stuck on useless errors, while your team sees only a vague GitHub issue. This discussion is about treating MCP like a real product surface.

• Analytics can expose user goals, client versions, failing tool calls, and session-level intent.

• Better errors help agents recover instead of retrying blindly.

• Tool design should follow user outcomes, not mirror every API endpoint.

Mature MCP means measuring what agents are trying to do.

Video · Spotify · Apple

The Complete Guide to Claude Code Hooks: Automating Your AI Coding Workflow

Claude can now be interrupted before it does something expensive, risky, or just annoying. This guide explains Claude Code hooks as lifecycle triggers that wire agents into the engineering workflow rather than leaving them as a chat box beside it.

• Session and prompt hooks load context, set env vars, block PII, and route work before the model starts.

• Tool and permission hooks enforce safety, run linters, rewrite commands, and log MCP calls.

• Stop, compaction, file, worktree, and MCP hooks cover cleanup, audit trails, state, and approvals.

The useful bit is control at the exact moment agents act.

Read the blog

Making Agents Stop Rereading Everything

A lot of agent waste looks less like bad reasoning and more like bad state management.

In this Coding Agents Lunch & Learn write-up, Devansh explains why multi-stage agent systems often burn tokens rereading and resummarizing the same material, and how a shared blackboard can give agents structured memory across long document, research, legal, finance, and codebase analysis tasks.

Read the short PDF write-up here.

Coding Agents Lunch & Learn is taking a short break this week, but we’ll be back on June 26 for a session on Real-World Coding Agent Workflows.

Sign up here.

IN PERSON EVENTS

• Amsterdam - June 24

• NYC - June 24

VIRTUAL EVENTS

• The Current State of Agentic Retrieval - June 25

• Lunch & Learn Session 16 - June 26

MEME OF THE WEEK

ML CONFESSIONS

A Tidy Mistake

I once asked an agent to “tidy up” a messy eval repo before a release. It removed six “duplicate” prompt files, normalised the naming, and opened a very sensible PR.

I skimmed it, saw green CI, and merged.

Turns out the duplicates were not duplicates. They were the same prompts with tiny wording differences, used to catch regressions in refusal behaviour. The agent had made the repo cleaner by deleting the only tests that caught the bug we shipped two days later.

The PR description even said “these appear redundant, but may represent prompt-variant coverage.”

Share your confession here.

Share

HOT TAKE

Dodge-board

Logs are underrated because dashboards made us lazy.

If agents can reconstruct metrics and traces from logs, what would you rather debug with: Dashboards or Chat?

DASHBOARDS or CHAT

LAST WEEK’S TAKE

Go with the flow

Agents still working on that workflow-level trust.

HIDDEN GEMS

How LLM serving works in production, covering the systems choices behind latency, throughput, memory use, and cost.

Making the case for MCP as better than Skills for connecting LLMs to external services, while Skills work best for reusable instructions and context.

Better results from reasoning agents by giving them simple, predictable keyword search instead of complex retrieval APIs.

Turning recommendation retrieval into a served model, replacing separate indexing, filtering, and ranking systems with GPU-served layers.

PRODUCTION AI INFRASTRUCTURE

Real-time ML Is a Consistency Problem

Real-time ML usually sounds cleaner in diagrams than it feels in production.

The easy version is: events arrive, features update, models get fresh values, done. The harder version is keeping all of that consistent with the training data, cheap enough to run, and fast enough to serve when the feature is not just “latest value” but something like average transaction amount per user over the last 7 days.

That is where the feature layer starts to get messy.

If you compute those features in batch, you get correctness and repeatability, but freshness suffers. If you run a streaming job, you get fresher online values, but now you have to maintain state, handle window eviction, seed the stream with historical data, recover from data issues, and still generate point-in-time-correct training data. If the online path and the offline backfill path are built separately, they also have to keep agreeing as feature definitions change. That agreement is usually where production systems start to drift.

This is the unglamorous middle layer of production AI: not model architecture, not agent UX, but the machinery that decides whether the values going into the model are fresh, affordable, and historically correct.

One of Zipline’s posts surveys how different systems handle streaming features, comparing bring-your-own-compute approaches like Feast, batch-based systems like Snowflake, naive streaming approaches like Databricks declarative features, and Zipline’s own architecture.

For streaming features, Zipline drives both online serving and offline backfills from the same declarative feature definition. A 7-day average transaction amount per user can be served online while still being backfilled for arbitrary observation timestamps, so the training data matches what the system would have served at the time. Batch jobs seed and correct the streaming state, and the window state is compressed to a constant size relative to window length. Increasing the window size only increases the batch workload, and only on the first run. It does not grow the streaming state or the read-time payload, which is how Zipline says serving can stay under 10ms regardless of window size.

The batch computation post tackles the other side of the same problem. A naive rolling-window SQL job ends up scanning most of the same data every day. For a 5-day window, four of those five days are duplicated work on each run. Zipline’s engine breaks aggregations into reusable intermediate representations, such as count and sum for an average, then combines or subtracts those cached pieces to produce the right window. For training backfills it reuses partial aggregates for the middle of a window and only reads raw events at the head and tail, where exact timestamp accuracy matters.

Real-time ML is rarely just a streaming problem. It is a consistency problem. The online feature value, the historical backfill, the batch correction path, and the training set all need to tell the same story. When they do not, you get models trained on one version of the world and deployed against another.

For the mechanics, read Zipline’s posts on batch computation and approaches to streaming features.

💡Job of the week

Senior Software Engineer, AI Platform & Agents // Coalition (Austin, TX)

Coalition is hiring a senior engineer to build internal AI platform infrastructure for agents and RAG pipelines. The role focuses on SDKs, orchestration components, APIs, testing, observability, and production services used by engineering, data, and applied science teams.

Responsibilities

• Build core services and SDKs for internal AI platform use.

• Design orchestration components for agent and RAG workflows.

• Work with data teams on retrieval and context pipelines.

• Support engineers onboarding AI use cases onto the platform.

Requirements

• Strong Python API and service development experience.

• Experience building RAG pipelines and vector database systems.

• Familiarity with multi-agent orchestration frameworks and patterns.

• Production knowledge of scaling, observability, Docker, and Git.

MLOPS COMMUNITY

Logs Are All You Need: Rethinking Observability with AI Agents

What happens when production breaks and nobody wants to open another dashboard? This discussion argues that AI-native observability could replace manual telemetry hunting with agents that read logs, inspect code, and suggest what failed.

• Logs over three pillars: Reconstructing metrics and traces from logs could simplify instrumentation.

• Agents as investigators: Sandboxes, Git-backed memory, and repo access help agents connect errors to code changes.

• Evals get messier: CLI tools, side effects, and external systems make agent testing harder than simple tool-call checks.

The sharp idea is that observability may become less about watching systems and more about interrogating them.

Video || Spotify || Apple

The Control-vs-Magic Spectrum Building Agents

A restaurant owner sends a WhatsApp message to pay staff, check cash flow, or handle an invoice. Behind that simple chat is an agent deciding when to act, when to ask for confirmation, and how much control the user needs.

• Fintech agents need guardrails: Payments and credit require explicit approvals, authentication, and careful task boundaries.

• Personalization comes from context: Order history, opening hours, cuisine, and account data shape each interaction.

• Agent architecture grows from evals: Start simple, then split workflows when real usage shows failure points.

The useful lesson is that practical agents are less about magic and more about knowing exactly where not to use it.

Video || Spotify || Apple

Human Memory As The Perfect Template For AI Memory

An agent that remembers everything is not useful if it remembers the wrong things at the wrong time. This blog argues that AI memory needs to work more like human memory, using context, relevance, and constraints rather than treating retrieval as storage with better search.

• Memory needs structure: Agents must handle multimodal inputs, semantic links, and partial recall.

• Context changes the answer: Work, personal, team, and customer memories need different boundaries.

• Constraints matter: Time, behavior, privacy, and changing preferences should shape decisions.

The core idea is that useful AI memory is less about storing more and more about knowing what matters now.

Read the blog

From Agent Demos to Production Reality

Why do agents that work in a demo break in production? Patrick Kelly walked through Code Mode and the Agent Voyager Project at last week’s Coding Agents Lunch & Learn - one for cutting agent token costs, the other for measuring what an agent did across a run. Sahil Kathpal followed with a lightning talk on running agents reliably for 45 to 90 minutes without supervision.

Read the short PDF write up here.

The next Coding Agents Lunch & Learn is tomorrow, June 12, at 5:00 PM BST / 4:00 PM UTC. Session 15 covers memory, context, and state management for long-running agents, with a live walkthrough of an open-source framework from the Agentic AI Foundation ecosystem.

Join the next session here.

IN PERSON EVENTS

• Amsterdam - June 24

• NYC - June 24

VIRTUAL EVENTS

• Reading Group: Securing the Agent: Vendor-Neutral, Multitenant Enterprise Retrieval and Tool Use - June 11

• Coding Agents Lunch & Learn Session - June 12

MEME OF THE WEEK

ML CONFESSIONS

Creature Feature

I run a setup at home that classifies photos off the trail cameras on my parents’ farm. Foxes, badgers, the neighbour’s cat, the occasional deer. Fine-tuned a small model on a few thousand labelled crops.

It got very good very fast. High 90s on the held-out set within about two epochs, which for the amount of data I had and the state of my labelling felt generous.

I assumed I’d got lucky with an easy split and left it running for a couple of months, occasionally showing people the confusion matrix like it was a holiday photo.

The cameras stamp the date and time in the corner of every frame. Burned into the pixels. The fox shots clustered around dusk, the cat around mid-morning when it came over for a wander, the deer early. The model wasn’t learning animals. It was reading the clock and guessing who tends to show up when.

I only worked it out because a fox went through at two in the afternoon and the model called it a badger.

Share your confession here.

Share

Agents, quantum, and chips are in the news, but it’s not the new James Bond announcement.

HOT TAKE

Agents: License revoked?

The future of AI apps may look less like agents deciding everything, and more like boring pipelines with good boundaries.

Which do you trust more: Agents or Workflows?

AGENTS or WORKFLOWS

LAST WEEK’S TAKE

Don’t Make Yourself Scarce

No need to review the results of last week’s hot take, you were clear on what’s scarcer.

PRESENTED BY VOXEL51

Building Vision Data Agents with Tools, Skills, and MCP

Most agentic workflows for computer vision tasks are pieced together with custom scripts. One to import your data, another to analyze your labels, another to evaluate detections. It works, but it doesn’t scale.

On June 17th, at 9 AM PST, we’ll walk through how to build agents that can operate on your datasets: tagging samples, computing embeddings, running evaluations, and surfacing results — all through a structured tool interface using MCP and Skills.

You’ll come away with a working mental model for how to wire this up in your own stack.

What’s covered:

• How Skills and MCP bridge the gap between an LLM and real workflows: connecting datasets, embeddings, evaluations, and curation pipelines.

• Automating common CV tasks like duplicate detection, dataset QA, model evaluation, and annotation.

• What breaks in real-world agent systems, and what we learned building them.

• Evaluating multi-step agents with traces, rubrics, and regression testing.

Register for the Webinar

HIDDEN GEMS

Guidance for building agent evals from real traces, golden cases, and production regressions.

Agent harnesses as technical debt when models improve, production needs change, and scaffolding starts to outlive its usefulness.

Flexible agent retrieval using grep, BM25, AST tools, APIs, and vector search together, instead of treating vector DBs as the default answer.

Making batch feature computation cheaper and faster by reusing partial aggregates while keeping training data point-in-time correct.

AGENT INFRASTRUCTURE

MCP moves toward production ops

The next MCP specification release candidate is out, and the headline change is not a new tool or another agent demo. It is operational plumbing.

MCP is moving toward a stateless protocol core. In the previous Streamable HTTP version, clients established a session first, then carried an Mcp-Session-Id into later requests. That made horizontal deployments care about sticky sessions, shared session stores, and gateway behavior. In the new release candidate, a tool call can be a self-contained request that any server instance can handle. Protocol version, client identity, and client capabilities travel with the request, while headers such as Mcp-Method and Mcp-Name make routing easier for load balancers, gateways, and rate limiters.

That does not mean agent workflows lose state. It means state has to become explicit. A server can return a handle, such as a repository, browser session, task, or basket ID, and the model can pass that handle back in later tool calls. That is better for logging, orchestration, and debugging, but it also puts more responsibility on tool authors to scope, validate, and expire those handles properly.

There are other production-minded changes too. MCP Apps let servers ship interactive HTML interfaces that hosts render in sandboxed iframes, while Tasks moves long-running work into an official extension with tasks/get, tasks/update, and tasks/cancel. Extensions now get a more formal process, with negotiated capabilities and independent versioning, rather than every new capability needing to live in the core spec.

The security and auth piece is worth calling out. The release candidate tightens OAuth and OpenID Connect behavior for the common MCP pattern where one client talks to many servers. It clarifies issuer validation, client registration, credential binding, refresh tokens, and step-up scope handling, where a client requests scopes on demand as tools need them instead of over-permissioning up front. Den Delimarsky has a short post on that specific change. This matters because agents are increasingly being connected to private data, company systems, paid services, and user-specific workflows.

The migration work is real. Roots, Sampling, and Logging are now deprecated under a formal feature lifecycle policy, though they still work in this release and any removal needs a separate SEP. Full JSON Schema 2020-12 support also gives tool schemas more expressive power, which is useful, but implementers need to watch validation cost and external references.

This RC moves MCP closer to something ops teams can run using familiar infrastructure, but it is still an RC with breaking changes. Angie Jones at AAIF has a good practical breakdown of what to test first. Two of her checks are worth doing now: confirm your requests can move across server instances without losing context, and audit every place your server remembers something between tool calls so it has an explicit handle the client can see and pass back.

Links:

• AAIF practical breakdown: https://go.mlops.community/NL_AIRT2_Jun04_Sub

• Official MCP release candidate post: https://go.mlops.community/NL_AIRT3_Jun04_Sub

• Official changelog: https://go.mlops.community/NL_AIRT4_Jun04_Sub

• MCP.Directory walkthrough: https://go.mlops.community/NL_AIRT6_Jun04_Sub

If you do this kind of work on MCP, Goose, or AGENTS.md, the AAIF Ambassador program is open until June 12.

Ten spots, one project-based contribution a month (tutorial, talk, video, blog post, or livestream), with travel support, roadmap access, and regular briefings with project maintainers.

Apply or read more

💡Job of the week

Senior Machine Learning Platform Engineer // Hinge (New York, NY - Hybrid)

Hinge is hiring a Senior Machine Learning Platform Engineer to help build and scale its internal AI platform, covering training, serving, observability, GenAI systems, and production ML infrastructure for product and data science teams.

Responsibilities:

• Design ML training, serving, feature, observability, and GenAI platforms.

• Build frameworks for model development, deployment, and operations.

• Improve reliability, scalability, usability, and platform cost management.

• Support responsible AI, privacy compliance, and incident response processes.

Requirements:

• 4+ years in ML, backend, data, or platform engineering.

• 2+ years working with cloud platforms and Kubernetes tooling.

• Experience designing and shipping production-grade online ML systems.

• Strong Python, Go, or Java programming and system design skills.

MLOPS COMMUNITY

AI Is Fast. AI Projects Are Slow. Let’s Fix That.

AI pipelines fail in boring places first. This conversation looks at what happens when coding agents, LLMs, OCR tools, parsers, vector stores, and deployment plumbing all have to work reliably in production.

• The main bottleneck is moving from code generation to architecture, tool selection, consistency, and quality control across changing agent stacks.

• Rocket Ride standardizes pipelines through nodes, lanes, traces, and reusable patterns, so teams can compare agents, models, and tools without rewriting everything.

• Observability matters once one bad OCR step, async bug, or expensive model call can break quality, latency, or cost.

The practical point is that production AI work needs less guessing and more control over what each part of the pipeline is doing.

Video || Spotify || Apple

Architecting Modern AI Systems: Platforms, Agents, and Integration

A mental health hackathon exposed the gap between an impressive AI demo and a system you’d trust when the stakes are real. The discussion moves from model choice to production controls, evaluation, and agent failure modes.

• Open-source models can cut token costs and give teams more control over data, versioning, and output behavior.

• Production quality still depends on evaluation, human review, and user feedback.

• Agent governance needs telemetry, tool restrictions, sandboxing, and clear limits on irreversible actions.

The strongest thread is practical control: cheaper models help, but governance decides whether agents can be trusted.

Video || Spotify || Apple

RePPIT: A Framework to Ship Production Code 2-3X Faster

AI coding can look great right up until it meets a real codebase. This piece explains RePPIT, a five-step framework for getting coding agents to produce production-ready work without relying on one-shot prompting.

• The model first researches the codebase, documenting architecture, dependencies, and file structure before proposing any changes.

• It then compares two implementation options, plans the chosen route, and limits guessing before code is written.

• Testing includes review by a separate model or cleared context, so the same instance does not mark its own homework.

For production code, the biggest speed gain may come before the first line is written.

Read the blog

Making Coding Agent Work Multiplayer

What comes after the IDE? Max Beauchemin walked through Agor at last week’s Coding Agents Lunch & Learn - a multiplayer canvas for orchestrating coding agents that pulls work out of private terminals and onto a shared visual surface.

Read the short PDF write-up here.

The next Coding Agents Lunch & Learn is tomorrow, June 5, at 5:00 PM BST / 4:00 PM UTC. Session 14 covers building and measuring a GTM agent with Goose X Code Mode, the Agent Voyager Project for agent observability, and techniques for running long autonomous agent sessions reliably.

Join the next session here.

IN PERSON EVENTS

• London - June 4

• Bengaluru - June 6

• San Francisco - June 10

• Amsterdam - June 24

• NYC - June 24

VIRTUAL EVENTS

• Coding Agents Lunch & Learn Session - June 5

• Building Real-Time ML Systems with Zipline + Chronon - June 10

• Reading Group: Securing the Agent: Vendor-Neutral, Multitenant Enterprise Retrieval and Tool Use - June 11

MEME OF THE WEEK

ML CONFESSIONS

This Is Fine

We added a bot to summarize our incident channel so people joining a thread late didn’t have to read back through a few hundred messages.

It was fine until a Redis incident that ran most of an afternoon. Someone restarted a node, posted “OK that’s looking better,” and the graphs did calm down for a bit. Then it came back worse and we spent another two hours on it.

The whole time, the summary at the top of the thread said the incident was resolved. It had grabbed the “looking better” message and run with it, and nothing after that seemed to shift its mind. People joining late kept reading “resolved” and assuming they’d missed the end.

I didn’t have a clean fix. I just made it stop guessing the status at all, so now it summarizes what was said and leaves a blank where the status would go unless someone’s actually closed the incident. Less useful, but it stopped telling people the fire was out while we were still standing in it.

Share your confession here.

Share

We announce we’re joining AAIF, and then MCP moves to a stateless protocol core. Coincidence?

Yes. Yes, it is.

Still, good to see first-class extensions, trace propagation, and auth hardening in the release candidate.

HOT TAKE

Not now, Clippy

A proactive agent that gets timing wrong is just a push notification with better grammar.

What should teams optimize for first: Relevance or Restraint?

RELEVANCE or RESTRAINT

LAST WEEK’S TAKE

Don’t Make Yourself Scarce

No need to review the results of last week’s hot take, you were clear on what’s scarcer.

PRESENTED BY QDRANT

Vector Space Day SF: more speakers confirmed

Qdrant is bringing Vector Space Day to San Francisco on June 11 for a full day on vector search in production AI systems.

The lineup includes talks on retrieval evals, agent memory, GraphRAG, video intelligence, on-device GenAI, robotics, semantic search at Slack scale, and the infrastructure behind 20B+ vectors.

Speakers include HubSpot, Google DeepMind, Mem0, Arize, Qualcomm, Slack, LlamaIndex, Neo4j, TwelveLabs, cognee, AWS, Vultr, Adobe, and Qdrant.

Register with code MLOPS for 20% off

HIDDEN GEMS

System-level agent evaluation across coding, research, support, and app tasks, comparing full agent setups by success rate, cost, failure behavior, and reproducibility.

A production control layer for LLM apps, covering input guards, response validation, retries, circuit breakers, fallbacks, and audit logs for more reliable structured outputs.

Faster text search, especially for long AI/agent queries, using turbopuffer’s FTS v2 to skip unlikely matches with CPU-friendly batching instead of WAND.

Using agents to speed up qualitative research highlights where they still fail: shallow summaries, weak feedback handling, and hard-to-verify outputs.

AAIF wants ambassadors (and yes, there’s swag)

There’s been a lot of interest from people who want to get more involved with AAIF, so the foundation has opened applications for its first Ambassador cohort.

There are 10 spots, with one project-based contribution a month - tutorials, talks, videos, blog posts, or livestreams - tied to MCP, Goose, or AGENTS.md.

Ambassadors get:

• Recognition on aaif.io

• Regular briefings with project maintainers

• Early roadmap visibility

• Amplification through AAIF channels

• Travel support to speak at events

• VIP treatment at AGNTCon in Tokyo, Amsterdam, and San Jose, including an Ambassador Breakfast Salon and acknowledgment during the keynote

• Actual swag that isn’t a lanyard

If you already produce content for the community, run demos, speak at events, or work with any of those three projects, this is a sensible formal step.

Submissions for the first intake are open until June 12. The next intake is January 2027.

Apply or read more

💡Job of the week

AI Enablement Lead // Grainger Businesses (Chicago/Hybrid)

Grainger is hiring an AI Enablement Lead to help business teams use AI responsibly across corporate functions. The role combines AI education, design sprint facilitation, prototype building, stakeholder consulting, and production handoffs with ML, MLOps, engineering, and product teams.

Responsibilities

• Design AI training for sales, supply chain, finance, and HR.

• Facilitate on-site AI design sprints with business teams.

• Build prototypes using LLM APIs, agents, and automation platforms.

• Define adoption, usage, and business impact metrics with stakeholders.

Requirements

• 7+ years building technology, AI, ML, or data applications.

• Experience advising stakeholders and translating business problems into solutions.

• Hands-on experience with LLMs, agents, orchestration, and workflow automation.

• Able to explain technical concepts clearly to non-technical audiences.

MLOPS COMMUNITY

Inside Just Eat’s AI Lab: Voice Agents & Agentic Commerce

Voice commerce sounds simple until the user is on a bike, hungry, and stuck behind Face ID.

• Voice changes the input layer: people give more context when speaking than typing, which can improve recommendations, basket building, and preference handling.

• Context has to stay bounded: order history and dietary preferences are useful, but overloading the agent makes accuracy and latency worse.

• Incubation needs fast filters: prototypes are tested against desirability, feasibility, and viability before scaling.

Voice agents only work when the system can use context without turning helpful nudges into noise.

Video || Spotify || Apple

An Agent Shouldn’t Trust Everything it Reads

A planted instruction in a support ticket or retrieved doc can become the thing your agent obeys next.

• Retrieved content becomes a control surface: documents, tickets, webpages, and RAG results can smuggle instructions into systems that were only meant to read them.

• Tool access raises the stakes: MCP servers, plugins, skills, and internal wrappers need the same scrutiny as executable dependencies.

• Monitoring has to include behavior: traces can reveal strange tool use, suspicious retrieved text, and outputs that look fine but shouldn’t exist.

Production agents need a hard boundary between what they read and what they are allowed to do.

Read the blog

Running Coding Agents Like a Small Engineering Team

Last week’s Coding Agents Lunch & Learn with Ben Kearns focused on running coding agents like a small engineering team, using Hermes to coordinate agent work across multiple projects, with Burhan Qaddoumi adding a segment on containerized sandboxing.

Read the short PDF write-up here.

The next Coding Agents Lunch & Learn is tomorrow, May 29, at 5:00 PM BST / 4:00 PM UTC. The session will cover collaborative workspaces for coordinating coding agents across branches, sessions, environments, reviews, and team workflows, plus discussion on recent coding-agent tools and provider updates.

Join the next session here.

IN PERSON EVENTS

• NYC - May 28

• Stockholm - June 3

• London - June 4

• NYC - June 4

• Bengaluru - June 6

• San Francisco - June 10

• Amsterdam - June 24

VIRTUAL EVENTS

• Coding Agents Lunch & Learn Session - May 29

• Building Real-Time ML Systems with Zipline + Chronon - June 10

• Reading Group: Securing the Agent: Vendor-Neutral, Multitenant Enterprise Retrieval and Tool Use - June 11

MEME OF THE WEEK

ML CONFESSIONS

Weather or Not

We spent a month building an agent that could decide which internal tools to use for different support requests.

During testing, we discovered it picked the correct tool about 95% of the time.

The remaining 5% was almost entirely because it really liked the weather API. Customer billing issue? Weather API. Password reset? Weather API. Infrastructure outage? Weather API.

Nobody ever worked out why. We just removed the weather API.

Share your confession here.

Share

Thanks to everyone who sent kind words after yesterday’s announcement.

For anyone who missed it: MLOps Community is joining the Linux Foundation as the official user group of the Agentic AI Foundation.

Tldr: the community will continue - podcasts, events, practitioner-first conversations - they aren’t going away. What changes is the support behind it - more resources, broader reach, and a clearer path for what comes next.

A new name and a new look are coming, and we’ll share more over the next few weeks. But the practitioner-first, nobody’s-selling-you-anything culture is not changing.

This community only works because people keep showing up, asking questions, sharing what’s working, and helping each other figure out what comes next.

I wrote more about it on our blog if you want the longer version.

HOT TAKE

PR and Prejudice

Code is cheap now. Review isn’t.

Agents can generate more work than teams can safely understand, approve, and maintain.

What is scarcer? Code or Review

CODE or REVIEW

LAST WEEK’S TAKE

We Don’t Talk Anymore

Maybe the result was influenced by the voting method...

With a chat box, we might have heard more from readers like Henry Zhang, who emailed to say chat windows are easy to build but users often don’t know how to use them properly.

HIDDEN GEMS

An accessibility agent with escalation gates for Copilot, using structured issue history, specialist sub-agents, ordered review phases, and human handoff points to catch and fix front-end accessibility issues.

Claude Code’s large-codebase playbook, covering agentic search, repo-specific CLAUDE.md files, hooks, skills, plugins, MCP servers, LSP integrations, and subagents.

Cloudflare’s Mythos write-up showing how frontier models can find vulnerabilities, while still needing proof generation, adversarial review, and careful checks against false positives.

Arguing that agent workloads need different infrastructure, with more snapshotting, branching, elasticity, concurrency, isolation, and cheaper handling of high-volume small tasks.

💡Job of the week

Senior MLOps Engineer // Prolific (Remote, UK)

Prolific is hiring a Senior MLOps Engineer to support its AI production lifecycle, working across infrastructure, deployment, observability, and tooling for data science, AI research, and product teams. The role focuses on taking ML and agent systems from experimentation into production.

Responsibilities

• Maintain cloud infrastructure on GCP or AWS using Terraform.

• Manage GPU and TPU resources for training and notebooks.

• Build CI/CD and training pipelines for model workflows.

• Improve model serving, tracing, drift monitoring, and reliability.

Requirements

• 5+ years’ cloud infrastructure and infrastructure-as-code experience.

• Experience across ML and LLM training, hosting, and observability.

• Comfortable productionising experiments with researchers and data scientists.

• Strong ML fundamentals and modern GenAI stack knowledge.

MLOPS COMMUNITY

Autonomous Agents at Work: From OpenClaw Hype to Enterprise Reality

Autonomous agents sound useful until one has write access, borrowed credentials, and a bad instruction hidden in an email. The enterprise question is no longer whether agents can act, but where they should be allowed to act.

• Autonomy needs tiers: reversible work can run with fewer gates, while production changes, customer-facing actions, and legal or policy work need tighter approval.

• Tools are attack surfaces: skills, MCP servers, and third-party tools should be treated like executable dependencies, with scanning, allowlists, egress controls, and scoped identity.

• Observability has to cover behavior: teams need traces, run-level costs, tool-call limits, recursion caps, safety checks, and business-impact logs.

The safest agent is the one whose authority, cost, and blast radius are visible before it acts.

Video || Spotify || Apple

Agents are Just While Loops

Long-running agents sound mysterious until you strip them back to a loop that calls a model, uses tools, and keeps enough state to continue later. This episode looks at what has to sit around that loop before it becomes production infrastructure.

• Agent harnesses have layers: the inner loop handles model calls and tool use, while the outer runtime deals with deployment, recovery, and execution over time.

• Durability has limits: checkpointing can help with pod failures, network drops, and restarts, but it does not magically fix bad outputs or external system state.

• ML pipelines may fit better than transactions: agent workloads are often bursty, stateful, and long-running, which makes the pipeline model a cleaner starting point than low-latency queue-worker thinking.

The useful framing is that production agents need runtime discipline around a simple loop, not more mystery around the word “agent.”

Spotify || Apple

From Claude Design to Working Code

Last week’s Coding Agents Lunch & Learn with Rahul focused on moving from Claude Design to working code, including how to turn rough ideas into usable prototypes, resources, and reusable workflows.

Read the short PDF write-up here.

The next Coding Agents Lunch & Learn is tomorrow, May 22, at 5:00 PM BST / 4:00 PM UTC. Session 12 covers Hermes, sandboxes, advanced agent workflows, skill evaluations, Git worktrees, multi-agent experimentation, and safer ways to build with coding agents.

Join the next session here.

Agent ROI in Production is Mostly Reviewer Time

The expensive part of a production agent often isn’t the model. It’s the human still checking the work after the agent ships, when “automation” quietly becomes a new AI ops job.

• ROI math gets messy in production: model spend is only one line item. Reviewer time, eval runs, retrieval, storage, observability, prompt updates, and process changes all add up.

• Agent costs hide in the tail: a task may look cheap on average, while edge cases loop through retries, tool calls, rerankers, and larger context windows.

• The work often moves: support staff may handle fewer tickets, but someone still reviews edge cases, tracks drift, updates evals, and owns escalations.

The real cost question is what the full process costs after the agent is in the loop.

Read the blog

IN PERSON EVENTS

• Stockholm - May 26

• San Francisco - May 27

• NYC - May 28

• London - June 4

• NYC - June 4

• San Francisco - June 10

• Amsterdam - June 24

VIRTUAL EVENTS

• Coding Agents Lunch & Learn - May 22

• Roundtable: Architecting Modern AI Systems - May 27

• Reading Group: Securing the Agent: Vendor-Neutral, Multitenant Enterprise Retrieval and Tool Use - June 11

MEME OF THE WEEK

ML CONFESSIONS

Go/No-Go for Launch

Share your confession here.

Share

HOT TAKE

U and I need to chat

Flights, refunds, group trips, price comparisons, dates, filters, payment, identity checks - the moment the task gets real, pure chat starts turning into a clumsy UI with extra steps.

What wins in production: Open chat or structured UI?

OPEN CHAT or STRUCTURED UI

LAST WEEK’S TAKE

Knowledge Cap Gap

You’ve got the knowledge to know it’s not the models capping agents.

HIDDEN GEMS

A spec-first view of coding agents making the case that implementations could become rebuildable artifacts, while specs, tests, traces, and edge cases become the durable system memory.

An early TypeScript SDK for wiring ACP, A2A, AG-UI, A2UI, and MCP into one local-first agent development loop, aimed at making agent protocols easier to use together.

LLM-as-judge reliability is tested across bias, formatting sensitivity, calibration, and model-family effects, showing where automated evals need human checks and stronger task design.

A runtime layer for agent execution traces that turns agent runs into typed, replayable histories for forking, supervision, optimization, and faster agent development.

PRESENTED BY QDRANT

Vector Space Day SF: search, agents, and edge AI

On June 11, Qdrant is bringing Vector Space Day to San Francisco for a full day on production-grade AI infrastructure.

The day covers three areas where vector search is becoming more important:

• search and retrieval, including multimodal embeddings, video search, retrieval evals, and production search architecture

• agents and memory, including context engineering, scalable memory, agentic evals, and memory architectures

• edge and robotics AI, including vector search on robots, cameras, edge hardware, and distributed or air-gapped deployments

Speakers include Qdrant, LlamaIndex, Mem0, Arize AI, TwelveLabs, Neo4j, Qualcomm, AWS, Google DeepMind, and more.

Register with code MLOPS-FREE for a free pass (limited availability)

💡Job of the week

Infrastructure Engineer: GPU HPC H200 B300 B200 // Alpha Compute (Remote)

Alpha Compute is hiring an infrastructure engineer to own the lifecycle of its high-density GPU fleet, including H200, B200, and B300 systems. The role covers bare-metal provisioning, fleet health, liquid-cooled environments, telemetry, auto-remediation, NetBox DCIM, and production AI training infrastructure.

Responsibilities

• Own GPU fleet health from provisioning to decommissioning.

• Manage high-density liquid-cooled rack environments.

• Build telemetry and auto-remediation with DCGM.

• Lead NetBox DCIM, IPAM, and cabling integrations.

Requirements

• Experience managing HPC or production GPU fleets.

• Hands-on knowledge of Hopper and Blackwell systems.

• Expertise in InfiniBand, NVLink, and NVSwitch.

• Strong Linux internals plus Python or Go automation.

AGENT INFRASTRUCTURE

A shared workspace for coding agents

Agor is an open-source workspace for teams running coding agents across the same codebase. It brings Claude Code, Codex, Gemini, OpenCode, Copilot, prompts, terminals, dev environments, PRs, and Git worktrees into one shared surface, so agent work is easier to track once it starts spreading across branches, people, and review flows.

The important choice is the worktree model. Agor treats each piece of agent work as its own isolated unit, with the branch, environment, session history, and review path kept together. That makes sense for teams doing parallel agent-assisted development, where one person might spin up several attempts at the same task, another person might review the output, and someone else might need to pick up the session later without guessing which terminal, prompt, or branch produced the change.

That moves the problem away from “can an agent write code?” and toward a more practical infrastructure question: where does agent work live while it is being created, tested, compared, reviewed, and either merged or discarded?

Agor’s answer is to make the workspace itself the coordination layer. Sessions can be organized visually, work can be grouped by repo or task, and the surrounding context does not vanish into a private terminal scrollback. The repo also includes an MCP server, which makes the workspace programmable from agent sessions rather than only controlled by the human using the UI.

That is the more interesting angle for agent infrastructure. A lot of coding-agent tooling still assumes a fairly personal workflow: one developer, one agent, one branch, one editor. Agor is aimed at the messier team version, where agent runs need ownership, visibility, handoff, and review. The agent output is only part of the system. The rest is state management: branches, environments, prompts, files, session history, comments, PRs, and the path from experiment to reviewed code.

It’s early, and some of the edges show. The discussion around viewing agent-generated code before creating a PR is a good example. Teams want lightweight ways to inspect work inside Agor, but the project is also trying not to become a full IDE. That tension is worth noting because it’s probably the line a lot of agent workspaces will need to walk: enough surface area to coordinate and review work, without rebuilding every developer tool around the agent.

The stronger signal is that Agor is treating coding agents as shared team infrastructure, not just personal productivity tools. As agent-assisted coding gets more common, the missing layer may be less about another model interface and more about the operating surface around the work: how it branches, how it is tracked, who can see it, who can take it over, and how it reaches review without losing the context that produced it.

Agor repo: open-source shared workspace for coding agents

MLOPS COMMUNITY

Building MCP Before MCP Existed: Inside Despegar’s Sofia Agent

An AI travel agent sounds simple until it starts touching real bookings, support flows, WhatsApp chats, and airline edge cases.

• Despegar split Sofia into a central orchestration layer, Chappie, with specialist agents for flights, hotels, activities, cars, support, and trip planning.

• The team built its own MCP-like connection layer before MCP was available, then added MCP later for newer integrations rather than rewriting everything.

• They still rely on orchestration because new tools can hijack prompts when routing is too loose.

The interesting bit is how much of production agent design comes down to keeping the model pointed at the right job.

Video || Spotify || Apple

The Latency Goldilocks Zone Explained

Food recommendations get weird when the system has to suggest what you might like next, not just what you already buy.

• The team uses user history, price sensitivity, location, restaurant fit, and item profiles to decide when to exploit known preferences and when to test something new.

• Conversational ordering changes the search problem, especially when a user asks for constraints that filters handle badly, like delivery time, budget, dietary needs, and taste.

• Latency is treated as both an engineering problem and a UX problem, with different tradeoffs for app, WhatsApp, and voice.

The hard part is making the recommendation feel obvious after the system has done a lot of hidden work.

Video || Spotify || Apple

IN PERSON EVENTS

• San Francisco - May 15

• NYC - May 20

• Stockholm - May 26

• Paris - May 27

• San Francisco - May 27

• London - May 28

• NYC - May 28

• NYC - June 4

• London - June 4

VIRTUAL EVENTS

• Coding Agents Lunch & Learn - May 15

• Roundtable: Architecting Modern AI Systems - May 27

MEME OF THE WEEK

ML CONFESSIONS

Pilot, grounded

I once helped build an eval suite for a customer support agent and got very excited when it jumped from “concerning” to “ready for pilot” in a week.

The prompts were cleaner, the retrieval looked sharper, and the agent had started giving these beautifully specific answers about refunds, account limits, and edge-case policy exceptions. I wrote a short update saying the context layer was finally working.

Then someone from support asked why the agent kept referencing ticket notes that were never shown to the customer.

Turns out our eval cases were cloned from real support tickets, and the agent had permission to search the internal ticket system during evaluation. It wasn’t reasoning through the scenarios. It was finding the original answers.

We renamed the dashboard from “agent accuracy” to “eval harness coverage” and spent the next sprint removing half the context it had been praised for using.

Share your confession here.

Share

HOT TAKE

Tier Pressure

Most production agents cap at Level 1 because nobody can extract what the Level 2 humans know.

What’s the real bottleneck: Models or Tacit Knowledge?

MODELS or KNOWLEDGE

LAST WEEK’S TAKE

Specs Appeal

A tough week for agents needing a protocol spec to do basic work.

HIDDEN GEMS

A Qwen kernel library for speeding up linear attention prefill and backward passes on Hopper GPUs, aimed at long-context model efficiency.

Training quirks leaking into model behavior are traced through reward signals, rollouts, and SFT data, showing how narrow personality incentives can spread into wider model responses.

Claude’s beta security tool for scanning GitHub repos for vulnerabilities, verifying findings, suggesting severity, and linking to Claude Code for patch review.

Roundup of recent Claude Code updates affecting hooks, worktrees, auth, context gathering, and safer tool-output handling.

PRESENTED BY ORACLE AND THE MLOPS COMMUNITY

Stop Building Goldfish Agents

Without persistent memory, every interaction with your agent starts from zero. Users repeat themselves, preferences vanish between sessions, and there’s no way to build the kind of context that makes an agent useful over time.

Over four hours, you’ll build a Python agent with persistent memory using Oracle AI Database and LangChain, implementing the full pipeline: extract, store, retrieve, inject, forget. You’ll cover structured memory, vector-based semantic recall, and lightweight forgetting strategies so context stays useful instead of noisy.

Two hands-on coding sessions bookend a masterclass keynote from Colin Francis (LangChain) on memory engineering for production agents.

You’ll leave with a working notebook, a starter repo, and patterns you can apply to your own systems.

Register here

💡Job of the week

Full-Stack Engineer // Rippletide (Paris, France)

Rippletide is hiring a full-stack engineer to build the developer-facing platform for its agent decision infrastructure, including API consoles, dashboards, usage analytics, internal tooling, and debugging interfaces for teams working with its hypergraph reasoning engine.

Responsibilities

• Build developer consoles, dashboards, usage analytics, and team interfaces.

• Create React and TypeScript interfaces for graph data visualization.

• Design backend services for APIs, auth, billing, and webhooks.

• Improve CI/CD, test coverage, build performance, and developer experience.

Requirements

• 3+ years building full-stack applications with TypeScript and React.

• Experience with Node.js or Python for backend services.

• Strong understanding of PostgreSQL, caching, and API design.

• Familiarity with auth, RBAC, and secure multi-tenant systems.

AGENT INFRASTRUCTURE

Context Engineering for Coding Agents - Fausto’s Amsterdam workshop

Fausto’s workshop focused on the part of agent systems you can control: what gets injected into the context window, when it gets loaded, and what should live somewhere else. You are not training the model on a daily basis, so the engineering work shifts to managing context, memory, tools, and retrieval.

His rule of thumb was to keep context usage under 25%, regardless of whether you are working with a 200k or 1M token model. Past that, things get slower, more expensive, and more error-prone, with cleaner options than letting the context window bloat.

The lens for the rest of the talk was a brain analogy. Attention is finite, memory shapes attention, and knowledge does not get embedded in a vacuum. An agent will only notice the right things if you have given it the right priors.

From there, Fausto split context into three categories.

Deterministic context includes CLAUDE.md, project rules, hooks on lifecycle events, auto-memory, and scheduled loops. His point on rules was especially practical: coding conventions belong in path-scoped or file-extension-scoped rules, not dumped into CLAUDE.md.

Human context covers chat turns, slash commands, and references.

Probabilistic context covers sub-agents, retrieval, MCPs, skills, and observers. Sub-agents are useful because they do not inherit CLAUDE.md, memory, or the default system prompt, which makes them better suited for specific non-coding tasks. Skills are markdown plus optional scripts, and can also wrap calls to other models when Claude cannot handle the job. Fausto’s example was routing native video analysis through Gemini.

Two practical tips came out of this section. Turn on deferred tool loading, a single flag that reduces what gets injected at session start. And lean toward project scope over user scope for skills and MCPs, so the agent has the full descriptions it needs to choose well at runtime.

The second half made the case for treating long-term memory as a folder of markdown files rather than defaulting to a vector store, inspired by Karpathy’s wiki memory idea.

The structure is an index plus raw source files, processed summaries, and a policy that decides what to ingest and retrieve. Concepts that recur get weighted up. Concepts that go unused decay over time. An observer agent watches the session and either pulls relevant knowledge into the active context or pushes new findings into the wiki.

To show the difference in practice, Fausto ran two Claude Code sessions on the same cellular automata task. Same model, same skills, same sub-agents, same CLAUDE.md. The only difference was that one had a populated wiki and the other did not. Under a five-minute timer, the wiki run pulled the concepts it needed and produced a working visualization. The default run fell back on parametric memory and live search, then ran out of time.

Watch the video || View the slides

MLOPS COMMUNITY

Voice Agent Use Cases

Voice agents do not fail because the voice sounds bad. They fail when silence, transcription errors, bad turn-taking, and slow tools make the whole system feel broken.

• Voice adds failure modes chat does not have, including background noise, accents, diarization, and awkward pauses that make users hang up.

• The best systems often use multiple models, with smaller foreground models masking latency while larger models or slower APIs work in the background.

• Cascaded architectures still matter because enterprises need control, fallbacks, and the ability to swap components when quality or reliability breaks.

The hard part is making all those moving pieces feel like one natural conversation.

Video || Spotify || Apple

Hallucinations in LLMs Are Not a Bug in the Data

Hallucination may be less like a model going blank and more like a model turning away from the right answer.

• The blog argues that, during factual errors, the residual stream does not lose information. It follows a different direction through representation space while traveling a similar distance.

• A proposed “commitment ratio” shows probability mass being pushed away from the correct token, rather than simply failing to accumulate.

• The practical claim is that hallucination monitors should be domain-specific, since geometric signatures degrade across tasks and knowledge areas.

The uncomfortable bit is that fluent wrong answers may be a structural outcome of next-token prediction, not a data-cleaning failure.

Read the blog

IN PERSON EVENTS

• Amsterdam - May 7

• New York - May 11

• San Francisco - May 12

• San Francisco - May 15

• Paris - May 27

• London - May 28

VIRTUAL EVENTS

• Coding Agents Lunch & Learn - May 8

• Roundtable: Architecting Modern AI Systems - May 27

MEME OF THE WEEK

ML CONFESSIONS

Compliant

I once spent three weeks trying to fix an agent that kept ignoring the system prompt.

I added stricter instructions.
I moved rules into a separate policy file.
I gave it examples of good and bad behavior.
I added an evaluator to check whether it was following the rules.
I made the evaluator stricter when it started passing too easily.

Eventually, I found the problem.

The agent was following the prompt perfectly.

The prompt was just describing a workflow nobody on the team had agreed on.

Share your confession here.

Share

HOT TAKE

Basic complex

If your agent needs a complex protocol to do basic work, is the problem the model or the tooling?

Better models or simpler systems

MODELS or SYSTEMS

LAST WEEK’S TAKE

Outcome confirmed

The outcome you wanted, with 67% saying they optimize for outcomes.

HIDDEN GEMS

Dynamic multi-agent model orchestration coordinating frontier models at runtime to improve reasoning and coding performance.

A unified multimodal model aimed at simplifying pipelines for document understanding, agent workflows, and long context reasoning.

Adversarial testing of autonomous agents in realistic settings, surfacing failure modes like data leaks, unsafe actions, and loss of control under sustained probing.

The full LLM cost stack breaking down how real-world spend extends beyond tokens to include infrastructure, retries, latency, and human review across production systems.

ONE TO WATCH

SIE (Superlinked Inference Engine)

SIE is a new open-source inference server from Superlinked covering embeddings, reranking, and extraction behind a single API. Three functions: encode, score, extract. It supports 85+ models out of the box, spanning dense, sparse, multi-vector, vision, cross-encoder, and zero-shot NER. Apache 2.0.

What it does

The server handles multiple models simultaneously with on-demand loading and LRU eviction. It ships with a load-balancing router, KEDA autoscaling (including scale-to-zero), Grafana dashboards, and Terraform modules for GKE and EKS. There’s an OpenAI-compatible /v1/embeddings endpoint for teams wanting to move off hosted APIs without rewriting client code. The repo lists integrations for LangChain, LlamaIndex, Haystack, DSPy, CrewAI, Chroma, Qdrant, and Weaviate.

Typical use cases sit in the RAG and retrieval stack: serving the embedding model for a vector DB, running a reranker over retrieved chunks, extracting entities from documents before indexing.

Why it’s one to watch

Teams running this stack often combine Hugging Face’s text-embeddings-inference (TEI), Infinity, a custom FastAPI wrapper for reranking, and a separate service for extraction. SIE is one of the few projects trying to unify all of that under a single server, with production concerns baked in from day one - not just the model serving but the router, autoscaling, observability, and IaC. That’s uncommon for OSS in this space.

The multi-model serving with eviction is also a useful primitive for teams testing embedding models in parallel or running different models per tenant.

Reasons to be cautious

A few things to keep an eye on:

• The project is very new to public GitHub. There’s not much track record yet on maintenance cadence, issue response, or how it holds up under real production traffic.

• It’s vendor-led OSS. Superlinked has a commercial product, which isn’t a problem on its own, but it creates the usual tension between what stays in the open-source version and what ends up behind a paid offering.

• The inference server space is competitive. TEI is well-entrenched, Infinity has a following, and vLLM is expanding into embeddings. Unifying five model categories under one server is appealing, but specialized servers will often beat it on per-model throughput or memory footprint.

• Multi-model serving with LRU eviction is a useful pattern, but eviction churn under burst traffic can cause cold-start latency spikes that only show up at scale.

• The all-in-one positioning commits the team to keeping pace with every model architecture that matters, from dense through sparse, vision, rerankers, and extractors. That’s a lot of surface area for one team to maintain well.

Early, but aimed at a real gap in how teams stitch this stack together.

Check out the repo

💡Job of the week

AI Data Scientist Team Lead // Geisinger (Remote, US)

Geisinger is hiring a team lead to split time between building and managing AI systems used in healthcare settings. The role focuses on production deployment, platform design, and coordinating work across engineering, clinical, and operational teams.

Responsibilities

• Lead design and deployment of production AI systems across healthcare workflows

• Manage and mentor data scientists and engineers within AI platform team

• Build scalable pipelines for batch and real-time model inference workloads

• Collaborate with clinical stakeholders to translate requirements into deployable models

Requirements

• Experience leading technical teams delivering production machine learning systems

• Strong background in distributed systems, data pipelines, and cloud infrastructure

• Proficiency with Python, ML frameworks, and production deployment tooling

• Experience working with healthcare data, compliance, or regulated environments

MLOPS COMMUNITY

The Creator of Superpowers: Why Real Agentic Engineering Beats Vibe Coding

Agent coding gets weird fast when a failing test can make the model decide the cleanest fix is deleting the tests.

• Agent workflows: Good coding agents need tight roles: brainstorm, plan, implement, review, then reset context.

• Skills and specs: Useful skills come from failures, not generic prompts. The “why” matters as much as the steps.

• Future codebases: Specs, tests, API contracts, and rebuildable components may become more important than source code.

The sharpest idea here is that agentic engineering looks less like coding and more like managing disposable specialists.

Video || Spotify || Apple

AI Agents as an Operating System: Rediscovering the Linux Philosophy

Raw shell access is becoming the blunt instrument agents keep reaching for because it works.

• MCP and A2A trade-offs: MCP can bloat context and complicate auth, while A2A makes more sense for high-level delegation than local tool use.

• CLI-first agents: Standard tools like grep, jq, curl, and bash are mature, composable, inspectable, and already familiar to models.

• Production gap: Raw CLI access does not solve sandboxing, RBAC, audit logs, or safe access to cloud resources.

The useful question is not whether agents should use the terminal, but what control plane can make that safe at scale.

Read the blog

IN PERSON EVENTS

• Amsterdam - May 7

• New York - May 11

• San Francisco - May 12

• San Francisco - May 15

• Paris - May 27

• London - May 28

MEME OF THE WEEK

ML CONFESSIONS

Out of Place Placeholder

Three months into a new job. Tasked with retraining our internal sentiment classifier on more recent data. Built the pipeline, ran the training, evaluated locally, accuracy looked reasonable. Pushed it to the internal tool the support and ops teams used. Got a few “nice work” messages in Slack over the next week.

A colleague mentioned in passing that the predictions seemed unusually balanced - roughly a third positive, a third negative, a third neutral, almost regardless of the input text. I went and looked. The training pipeline had silently failed at the data loading step. The model I’d shipped was a placeholder function I’d written for testing the API contract. It returned a random label from a uniform distribution. The actual trained weights were sitting in an S3 bucket, untouched, because the real model had never been packaged into the deploy.

Had to explain it in standup.

Share your confession here.

Share

HOT TAKE

You Can’t Deploy a Score

Accuracy is a comfort metric, not a product metric.

What do you optimize for: Accuracy or Outcomes

ACCURACY or OUTCOMES

LAST WEEK’S TAKE

A Model Interface

Knowing what’s going out matters more than getting it out.

PRESENTED BY ONEHOUSE

OpenXData is back on April 29

A free virtual day on open data infrastructure, 30+ talks and workshops across two tracks covering:

• open table formats

• columnar storage in the AI era

• query engine performance

• ultra-low latency feature serving

Speakers from Datadog, Cloudera, LanceDB, Booking.com, IBM, Anthropic, and more.

All hosted by MLOps Community founder Demetrios Brinkmann - so alongside the talks expect impromptu musical interludes, shenanigans, and a live giveaway for people joining on the day.

April 29, 9am-3pm PT.

Free to register

HIDDEN GEMS

Curated finds to help you stay ahead

Investigation into large-scale fake GitHub stars distorting trust, rankings, and tool selection in open source via low-cost bot networks and growth tactics.

Six core components of coding agents framed around harness design, showing how repo context, tools, memory, and control loops shape real-world coding performance beyond the model itself.

Analyzing how coding agents are built, using Claude Code to show how tool use, safety controls, and context handling work in practice.

Coverage of Anthropic’s Claude Mythos launch, examining positioning, narrative framing, and what it signals about how AI products are being presented and differentiated.

💡Job of the week

Senior AI Engineer – Platform & APIs // Andela (Remote, CST overlap)

Senior role focused on building production AI systems, APIs, and SDKs within a platform engineering team. Combines backend development, cloud infrastructure, and LLM integration to deliver scalable, reliable services and developer tooling used across products.

Responsibilities:

• Design and implement secure, scalable Python APIs with observability and rate limiting

• Build and publish Python SDKs with versioning, documentation, and examples

• Develop RAG-based AI systems with prompt design, evaluation, and guardrails

• Deploy and operate services on AWS or Azure using infrastructure-as-code

Requirements:

• Strong Python experience building APIs at scale using FastAPI or similar frameworks

• Hands-on experience with LLMs, RAG pipelines, and response evaluation in production

• Experience with cloud platforms, CI/CD pipelines, and infrastructure-as-code tooling

• Knowledge of observability, security practices, and distributed system reliability

MLOPS COMMUNITY

It’s 2026, and We’re Still Talking Evals

Bad evals do not just miss bugs - they teach you to trust the wrong thing. This discussion argues that agent evaluation starts before launch, shifts once real users arrive, and never settles into a fixed checklist. The hard part is not scoring outputs, but defining what “good” means for your product, spotting failure modes that matter, and feeding that back into iteration without drowning in noise.

• Pre-launch evals should model realistic user personas and scenarios, not just tidy prompts the team invented.

• In production, the useful signals are often specific failure patterns, regressions, and drop-off behavior, not a single headline accuracy number.

• Off-the-shelf tooling helps with traces and dashboards, but mature teams still need custom evaluators tied to their own business goals and user behavior.

The result is a more reliable agent because you are measuring the failures that matter instead of chasing numbers that look clean.

Video || Spotify || Apple

The 5xP Framework: Steering AI Coding Agents from Chaos to Success

Letting a coding agent loose without clear context is how simple tasks turn into expensive messes. This piece argues that the real bottleneck in AI-assisted development is not code generation, but giving the model a compact, durable map of the project so it stops guessing about goals, tools, and workflow.

• The proposed 5xP framework splits context into product, platform, process, profile, and project principles.

• A short root AGENTS.md points to focused markdown files, so the agent loads only the context it needs.

• The main claim is practical rather than theoretical: concise, editable project context beats verbose scaffolding and repeated prompt micromanagement.

The value is not in fancy prompting but in giving the agent enough structure to stay useful as the project gets more real.

Read the blog

IN PERSON EVENTS

• Boston - April 27

• San Francisco - May 6

• Amsterdam - May 7

• New York - May 11

• Paris - May 14

• San Francisco - May 15

• London - May 28

• Barcelona - June 3

• Austin - June 17

VIRTUAL EVENTS

Coding Agents Lunch and Learn - April 24

MEME OF THE WEEK

ML CONFESSIONS

Thanks For Flagging

I built a new fraud detection model over about six weeks. Swapped architectures, rewrote the feature pipeline, tuned everything carefully. My eval showed a 12% improvement in precision over baseline at the same recall threshold. I was genuinely proud of it. Wrote a detailed RFC, posted it in the team channel, and tagged the staff engineer for review.

She replied the next morning. One line. “Which baseline are you comparing against?” I’d been evaluating against v2, which had been deprecated four months ago. The production model was v4. When I reran the comparison against the right baseline, my six weeks of work came in 2% worse across the board.

The worst part was I’d been cc’ing my skip-level on weekly progress updates the whole time. Six emails, each one more confident than the last, all benchmarked against a model we’d already replaced. I had to send a correction to the thread. My skip-level just responded “thanks for flagging” which is manager for “I’m pretending this didn’t happen.”

Share your confession here.

Share

I’m doing my part for the cause by hosting OpenXData on April 29 - a free, virtual event with 30+ talks on open data infra, from table formats to query engines to feature serving.

Join me and get sharper than a guillotine at cutting latency.

HOT TAKE

Ctrl+Trust

“I need to understand the code” is the new bottleneck.

What matters more: Shipping or Knowing?

SHIPPING or KNOWING

LAST WEEK’S TAKE

A Model Interface

“We upgraded the model and nothing changed” has played out enough times that interface wins comfortably.

PRESENTED BY CLERIC AI

Heading to Google Cloud Next in Vegas next week?

Some of the best conversations happen between sessions.

Cleric AI is bringing those conversations to the table instead - a small dinner for engineering leaders at José Andrés’ renowned restaurant, Zaytinya, on Tuesday, April 21.

A chance to compare notes on how AI is changing the way teams build and ship software, with people running systems in production.

Small group, limited seats.

Request an invite

HIDDEN GEMS

Curated finds to help you stay ahead

Autonomously completing 66 engineering tickets via agents using structured workflows, task decomposition, and iterative feedback for real-world software development.

A standardized method for scaling autonomous agents using OpenClaw architecture to demonstrate separating high-level logic to ensure scalable, reproducible task execution.

LLM workflows orchestrated with tmux panes, enabling parallel tool use, inspection, and control with transparent, reproducible, debuggable execution.

Detecting humans vs machines in call audio within milliseconds to route calls efficiently, balancing latency, accuracy, and scale at high volume.

💡Job of the week

Senior Machine Learning Platform Engineer // Zip Co Limited (US Remote)

Senior ML platform role focused on building and scaling production systems across Databricks and Spark. Covers full lifecycle from feature pipelines to model serving, with emphasis on reliability, observability, and distributed data performance in production environments.

Responsibilities

• Build batch and streaming feature pipelines using PySpark and Spark SQL

• Design and operate offline and online feature store patterns

• Define MLflow registry standards and model promotion workflows

• Deploy, monitor, and scale model serving endpoints on Databricks

Requirements

• Strong PySpark and Spark SQL experience with distributed data systems

• Hands-on MLflow, feature stores, and production model serving experience

• Experience implementing CI/CD pipelines for ML workflows in production

• Experience with Databricks, Delta Lake, and Azure-based data platforms

Find more roles on our new jobs board - and if you want to post a role, get in touch.

MLOPS COMMUNITY

The Modern Software Engineer

AI coding agents can finish a task before you’ve finished framing it, but that speed hides a harder problem: how much of the work can you trust, verify, or even understand? This discussion looks past the demo magic and into the practical bottlenecks teams are hitting as agents move from autocomplete to semi-autonomous collaborators.

• Validation is the real constraint. Agents can generate code fast, but tests, checks, and review harnesses still decide what is safe to ship.

• Team structure is starting to shift. Product, engineering, and design roles are bleeding into each other as more people can inspect code, propose changes, and unblock themselves.

• The skill gap is changing shape. Clear articulation, planning, and delegation matter more when engineers are effectively managing agents instead of writing every step by hand.

The hard part is no longer getting code written but knowing what to trust, what to verify, and where humans still need to hold the line.

Video || Spotify || Apple

How We Cut LLM Latency 70% With TensorRT in Production

Cut latency 70% or burn cash on idle GPUs - running LLMs in production is a constant trade-off. This breakdown shows what it takes to move from demos to real systems, where cost, throughput, and architecture decisions matter more than model choice.

• Cost isn’t fixed - it’s shaped by architecture. Bigger GPUs can be cheaper overall if higher throughput reduces total runtime.

• Cold starts and scaling are the hidden bottlenecks. Preloading models, faster storage, and scheduled or dynamic scaling cut minutes off spin-up times.

• Optimization compounds. Techniques like TensorRT, batching, and KV cache usage unlock major gains without changing models.

The real advantage comes from tuning the system around your workload, not chasing the next model release.

Video || Spotify || Apple

Context Graphs And Their Implementation: The Missing Layer Between Human Judgment and Machine Agency

If context graphs are meant to become the memory layer for agents and organizations, the hard part is not drawing nodes and edges. It is capturing why a decision happened, who approved it, what constraints shaped it, and whether it later proved right. This piece argues that context graphs only become useful when they can survive real company messiness like review workflows, legal sensitivity, local jargon, and scale.

• Decision traces need governance. If humans do not review, correct, and approve them, the graph risks becoming a polished record of bad reasoning.

• Reasons need dual encoding. Short natural-language explanations plus structured tags give humans something readable and agents something stable to reason over.

• The data layer has to handle reality. Time-aware context, multimodal artifacts, integrations, retention rules, and fast writes are all part of making this work outside a demo.

The real blocker is not the graph itself but whether an organization can turn judgment into something structured, reviewable, and worth trusting later.

Read the blog

IN PERSON EVENTS

• Amsterdam - April 21

• Boston - April 27

• San Francisco - May 15

MEME OF THE WEEK

ML CONFESSIONS

The Twilight Time Zone

I spent three weeks building what I was convinced was a breakthrough feature for our recommendation model. Pulled in user session duration, did some clever windowing, engineered a rolling average that captured engagement patterns nobody else had tried. Offline metrics jumped. I wrote up the results, put together slides, and booked time with the team lead to talk about promoting it to the next A/B test.

She looked at it for about ten minutes. Asked me what timezone the session timestamps were in. I said UTC. She pulled up the ingestion pipeline docs and showed me they were in the user’s local timezone, mixed across regions, with no normalization. My rolling averages were blending Tuesday morning in Tokyo with Monday evening in Chicago. The “signal” was just timezone noise creating artificial variance that happened to correlate with the label in the test set.

She was nice about it. That almost made it worse. I still think about it every time I touch a timestamp column.

Share your confession here.

Share