Reliability

MCP gives you a standard wire format for connecting an agent to tools — it doesn’t make the tools on the other end safe to call. Most MCP servers are thin wrappers around an existing REST API, which quietly imports every assumption that API made about having a human developer as its client. Here’s what MCP actually is, how it differs from the API you already know, and the specific tool-design mistakes that turn a working MCP server into a flaky agent.

Idempotent means doing something twice has the same effect as doing it once. This is essential for APIs, agents, and any system where retries can happen. Here’s why, and how to get it right.

The per-token rate on Anthropic’s pricing page is real, but it’s the least useful number for predicting your actual invoice. Three multipliers move the bill far more than model choice does — a cache that silently misses, a transcript that grows quadratically with run length, and a best-of-N pattern that prepays for latency you may not need. Here’s the arithmetic on all three, combined, with real numbers.

An agent called an API, got a syntactically valid response, trusted it, and built three wrong decisions on top of it. No error was thrown. Every step succeeded locally. The cascade cost was 15× the original bad call — and all of it was preventable by a two-line validation check. This is validation debt: paying the cost of skipped checks in compounding failure downstream.

Launching N attempts at once and keeping the first success feels like a free win over sequential retries — you trade money for tail latency, and money is supposedly the thing you have more of. It isn’t free, and it isn’t even always a trade: for the correlated failures that dominate real production incidents, best-of-N pays for N guaranteed-identical failures up front instead of stopping at one.

A user closes the tab. An upstream request times out. A parent agent gets cancelled by its own budget. None of that reliably reaches the retry loop three calls deep, so the retry keeps going — burning tokens and rate-limit headroom for a result nobody will ever read. Cancellation is the one signal every fleet retry pattern assumes exists and almost none actually propagate.

A 429 and a 500 both land in the same except block, so most retry budgets treat them the same: one bucket, one backoff curve, one circuit breaker. That conflation is wrong in both directions — it makes you wait too little for the failure that isn’t yours, and panic too much over the one that is. Two error classes, two buckets, and the Retry-After header everyone reads and no one obeys.

You built an LLM judge to grade your agent. What grades the judge? An eval you never validated is a ruler you never checked against a meter stick — and a biased judge doesn’t just add noise, it moves your headline number in a consistent direction. How to meta-evaluate a judge: the labeled set, the agreement metric that isn’t accuracy, and the drift check.

‘When not to build an agent’ made the case against the loop in the abstract — quadratic cost, serial latency, an untestable failure surface. This is the concrete follow-on: three fixed pipeline shapes (linear chain, router-plus-handlers, fan-out/fan-in) that cover most of what people default to a loop for, why each one is cheaper and more testable, and the one test for when a real loop actually earns its cost.

Retry budgets cap HOW MUCH you retry; this is about WHEN to retry at all. The decision isn’t uniform: user-facing operations, background jobs, and fleet-wide cascades each have different failure costs, different retry ceilings, and different layers where the decision lives. A cost model for when to fail fast instead.