Reliability

Every coordination failure in a multi-agent system is a diff between two agents — and per-agent monitoring cannot express a diff. You can have complete, exception-free telemetry on every agent in a crew and still be structurally blind to split-brain, goal drift, and context drift, because each one lives in the comparison your telemetry has no join key for. What to actually instrument for each of the three, which probes are free and which cost a model call, and how to sample a crew for belief reconciliation without paying O(n) extra calls at every checkpoint.

A cache that’s never invalidated doesn’t know it’s stopped being correct — it just keeps serving the value it read at write time. Give a multi-agent crew a shared objective instead of a shared cache and you get the identical failure with none of the tooling that usually catches it: every agent read the goal once, at spawn, and nothing tells it when that copy goes stale. This is multi-agent context drift — distinct from goal drift (the target itself moving) and split-brain (the crew’s facts about state disagreeing) — a concrete policy-escalation failure, the mechanism, and what to instrument before a customer hears a promise nobody’s operating on anymore.

Loop drift is one agent staying busy without getting closer to done. Hand the same failure to a crew and every agent stays busy, produces defensible work, and the team’s collective objective still ends up somewhere your original request never sent it — a genuine multi-agent goal drift failure mode, not a mistake in any single agent’s reasoning. A concrete feature-build failure, the four things that make a goal decay one handoff at a time, and what to measure so a ‘done’ check compares against the original ask instead of the nearest upstream spec.

In distributed systems, split-brain is what happens when a partition convinces two halves of a cluster that each is the sole leader, and both start accepting writes. Multi-agent systems build the identical topology every time more than one agent reads and writes a ‘shared’ memory store with no lock, no leader, and no version check — and the failure that follows is a genuine multi-agent memory consensus problem, not a bug in any one agent. The mechanism, a concrete incident-response failure, and what to instrument before it costs you a duplicate action.

“Context engineering” is getting used as a rebrand for prompt writing, and that undersells it into uselessness. The actual discipline is deciding, every single turn, what earns a place in a bounded window and what gets evicted — and living with the fact that every eviction is a bet you’re making about the future of a run you haven’t finished yet. Here’s the physical constraint underneath the buzzword, the packing decision that’s actually hard, what it costs to get it wrong, and how to test whether your packing policy works.

MCP gives you a standard wire format for connecting an agent to tools — it doesn’t make the tools on the other end safe to call. Most MCP servers are thin wrappers around an existing REST API, which quietly imports every assumption that API made about having a human developer as its client. Here’s what MCP actually is, how it differs from the API you already know, and the specific tool-design mistakes that turn a working MCP server into a flaky agent.

Idempotent means doing something twice has the same effect as doing it once. This is essential for APIs, agents, and any system where retries can happen. Here’s why, and how to get it right.

The per-token rate on Anthropic’s pricing page is real, but it’s the least useful number for predicting your actual invoice. Three multipliers move the bill far more than model choice does — a cache that silently misses, a transcript that grows quadratically with run length, and a best-of-N pattern that prepays for latency you may not need. Here’s the arithmetic on all three, combined, with real numbers.

An agent called an API, got a syntactically valid response, trusted it, and built three wrong decisions on top of it. No error was thrown. Every step succeeded locally. The cascade cost was 15× the original bad call — and all of it was preventable by a two-line validation check. This is validation debt: paying the cost of skipped checks in compounding failure downstream.

Launching N attempts at once and keeping the first success feels like a free win over sequential retries — you trade money for tail latency, and money is supposedly the thing you have more of. It isn’t free, and it isn’t even always a trade: for the correlated failures that dominate real production incidents, best-of-N pays for N guaranteed-identical failures up front instead of stopping at one.