Launching N attempts at once and keeping the first success feels like a free win over sequential retries — you trade money for tail latency, and money is supposedly the thing you have more of. It isn’t free, and it isn’t even always a trade: for the correlated failures that dominate real production incidents, best-of-N pays for N guaranteed-identical failures up front instead of stopping at one.
A user closes the tab. An upstream request times out. A parent agent gets cancelled by its own budget. None of that reliably reaches the retry loop three calls deep, so the retry keeps going — burning tokens and rate-limit headroom for a result nobody will ever read. Cancellation is the one signal every fleet retry pattern assumes exists and almost none actually propagate.
A 429 and a 500 both land in the same except block, so most retry budgets treat them the same: one bucket, one backoff curve, one circuit breaker. That conflation is wrong in both directions — it makes you wait too little for the failure that isn’t yours, and panic too much over the one that is. Two error classes, two buckets, and the Retry-After header everyone reads and no one obeys.
You can’t tune a retry strategy you can’t see. This is how to instrument retries so you know whether each one succeeds (saving you money) or fails permanently (burning it). Three structured metrics turn a blind spot into actionable signal.
The retry decision (when to give up, how long to wait) only works if you can see inside it. Idempotency keys, retry-attempt correlation, and cost accounting are the foundation. Without them, your retry logic is flying blind — you can’t tell if you’re fixing a transient failure or burning your budget on a permanent one.
Retry budgets cap HOW MUCH you retry; this is about WHEN to retry at all. The decision isn’t uniform: user-facing operations, background jobs, and fleet-wide cascades each have different failure costs, different retry ceilings, and different layers where the decision lives. A cost model for when to fail fast instead.
The default retry pattern — catch the failure, append ’that didn’t work, try again,’ resend the same messages — doesn’t give the model a clean second attempt. It gives the model a context window containing its own wrong answer, which is exactly the thing most likely to make the second attempt rhyme with the first. Why retries poison the window, how to tell a poisoning retry from a safe one, and a scrub step that keeps the constraints without keeping the wrong path.
A retry budget is a language-agnostic idea, but the place you enforce it is not. Python’s tenacity decorators, Go’s context-plus-backoff, and JavaScript’s promise chains each make a different mistake easy and a different guarantee hard. Where the shared budget lives, and the per-language trap that leaks it.
A per-step retry cap bounds a step. It never bounds a run, and it never bounds a fleet — twelve workers each retrying ‘reasonably’ is how you turn one bad deploy into a bill. The four patterns that actually put a ceiling on what a fleet of agents can spend recovering from a failure: shared retry budgets, circuit breakers, decorrelated backoff, and poison quarantine.
Retrying a read is free. Retrying a write can charge a card twice, send two emails, or book two rooms — and the model has no idea it happened. Retry safety is a property you build into the tool, not a flag you set on the loop. Here’s why at-least-once delivery is the default you’re actually running, how to derive a stable idempotency key from an agent’s intent, and a dedup wrapper that makes any write safe to retry.