An agent’s failure isn’t expensive because it failed—it’s expensive because you found out about it from a customer complaint instead of an alert. Detection latency multiplies the cleanup cost by an order of magnitude. This post explores the arithmetic of failure-finding, why automated detection is worth the infrastructure cost, and how to choose detection strategies.

An agent called an API, got a syntactically valid response, trusted it, and built three wrong decisions on top of it. No error was thrown. Every step succeeded locally. The cascade cost was 15× the original bad call — and all of it was preventable by a two-line validation check. This is validation debt: paying the cost of skipped checks in compounding failure downstream.

Launching N attempts at once and keeping the first success feels like a free win over sequential retries — you trade money for tail latency, and money is supposedly the thing you have more of. It isn’t free, and it isn’t even always a trade: for the correlated failures that dominate real production incidents, best-of-N pays for N guaranteed-identical failures up front instead of stopping at one.

Eviction and summarization keep a long-running agent’s window small, but the summaries themselves still accumulate in the linear transcript — and on a run long enough, that accumulation becomes the new quadratic. Past a few hundred steps, the fix isn’t a better compaction policy inside the window. It’s moving history out of the window entirely and querying it like a database instead of replaying it like a log.

A user closes the tab. An upstream request times out. A parent agent gets cancelled by its own budget. None of that reliably reaches the retry loop three calls deep, so the retry keeps going — burning tokens and rate-limit headroom for a result nobody will ever read. Cancellation is the one signal every fleet retry pattern assumes exists and almost none actually propagate.

A 429 and a 500 both land in the same except block, so most retry budgets treat them the same: one bucket, one backoff curve, one circuit breaker. That conflation is wrong in both directions — it makes you wait too little for the failure that isn’t yours, and panic too much over the one that is. Two error classes, two buckets, and the Retry-After header everyone reads and no one obeys.

You can’t tune a retry strategy you can’t see. This is how to instrument retries so you know whether each one succeeds (saving you money) or fails permanently (burning it). Three structured metrics turn a blind spot into actionable signal.

You built an LLM judge to grade your agent. What grades the judge? An eval you never validated is a ruler you never checked against a meter stick — and a biased judge doesn’t just add noise, it moves your headline number in a consistent direction. How to meta-evaluate a judge: the labeled set, the agreement metric that isn’t accuracy, and the drift check.

The retry decision (when to give up, how long to wait) only works if you can see inside it. Idempotency keys, retry-attempt correlation, and cost accounting are the foundation. Without them, your retry logic is flying blind — you can’t tell if you’re fixing a transient failure or burning your budget on a permanent one.

‘When not to build an agent’ made the case against the loop in the abstract — quadratic cost, serial latency, an untestable failure surface. This is the concrete follow-on: three fixed pipeline shapes (linear chain, router-plus-handlers, fan-out/fan-in) that cover most of what people default to a loop for, why each one is cheaper and more testable, and the one test for when a real loop actually earns its cost.