A retry budget is a language-agnostic idea, but the place you enforce it is not. Python’s tenacity decorators, Go’s context-plus-backoff, and JavaScript’s promise chains each make a different mistake easy and a different guarantee hard. Where the shared budget lives, and the per-language trap that leaks it.
The cheap part of a failed agent run is running it again. The expensive part is that you can’t — the failure was non-deterministic, so the run that broke is gone, and you pay to summon it back. A cost model shows why reproduction, not repair, dominates your debugging bill, and why always-on tracing is almost always cheaper than the alternative it replaces.
Everyone budgets the token bill because the provider hands you an invoice for it. But an agent in production spends across five other axes that never show up on that invoice — wall-clock latency, orchestration, tool-call fees, human review, and idle polling — and for a lot of workloads the tokens are the smallest line. A model that sums all six so you can see which one you’re actually paying.
Most requests to your agent are easy, and you’re paying frontier-model prices for all of them anyway. A routing cascade — try the cheap model, escalate on a measurable confidence signal — cuts spend without touching output quality, if you get the escalation trigger right. Here’s the pattern, where it breaks, and the arithmetic on when it’s worth building.
A per-step retry cap bounds a step. It never bounds a run, and it never bounds a fleet — twelve workers each retrying ‘reasonably’ is how you turn one bad deploy into a bill. The four patterns that actually put a ceiling on what a fleet of agents can spend recovering from a failure: shared retry budgets, circuit breakers, decorrelated backoff, and poison quarantine.
A naive agent’s token bill doesn’t grow with the number of steps — it grows with the square of them, because every step re-reads the whole transcript that every previous step appended to. A small cost model shows the curve, and four structural moves turn the quadratic back into something close to linear without dropping information the agent actually needs.
An agent burned ~$200 overnight retrying an HTTP 400 — a request that was defined to fail. No component was buggy; each layer retried “reasonably.” The teardown: why retryability is a property of the error and not a default, how three nested retry caps multiply into 75 doomed attempts per item, and why per-step caps never bound a bill. With the two-line fix and a circuit breaker.
An agent is an LLM that controls its own control flow — and that autonomy has a price you pay on every run: quadratic token cost, serial latency, and a failure surface you can’t unit-test. Most tasks people reach for an agent on are a fixed pipeline wearing a costume. Here’s the decision checklist I use, the arithmetic on what the agent tax actually costs, and the same task built both ways so you can see the difference.
Treating the context window as append-only memory is how agents get slow, expensive, and quietly wrong. The fix is to run it like a cache with a budget and an eviction policy: decide what earns its tokens every turn. Here’s the cost math, the accuracy failure mode, and a working context manager.
Retries feel cheap and local. In a multi-step agent they’re neither. A small cost model shows why 20% per-step failure can more than double your bill — and how your recovery architecture, not your failure rate, decides the multiplier.