If your agent retries a failed step, you probably budgeted for it as 20% more failures, 20% more cost. That intuition is off by roughly double, and in one common architecture it’s off by a factor of three. This post works out the actual number, with a model you can run yourself.

That gap — between the cost you reasoned about and the cost you got — is most of why I started writing here. A demo agent never shows you this. It runs once, the retry path barely fires, and the bill is a rounding error. Then you ship it, real inputs push the per-step failure rate up, and the retry math that was invisible on stage becomes the line item your finance team asks about. This blog is for that second phase: the part where the agent survives contact with production and then does something expensive.

The naive model, and why it’s wrong

Here’s the intuition to kill. You have an agent that takes N steps to finish a task. Each step is a model call plus a tool call. Some steps fail — a tool returns an error, the model emits malformed arguments, a downstream API times out — and you retry them. If each step fails independently with probability p, surely the cost overhead is about p: 20% failure, 20% more spend.

Two things break that. First, a retry is not a cheap local do-over. In an agent the transcript grows every step, and every model call re-reads the entire transcript so far as prefill. Retrying step 12 doesn’t cost one step’s worth of tokens; it costs twelve steps’ worth, because that’s the context you re-send. Second, failed attempts don’t vanish. The model’s bad output and the error observation usually stay in the window — the model needs to see what went wrong to recover — so a single failure inflates the prefill of every step that follows it, not just its own retry.

Retries, in other words, are multiplicative and coupled to context growth. Let’s measure how much that matters instead of hand-waving.

A cost model you can run

This is deliberately small. It counts tokens across a multi-step agent run, prices input (prefill) and output separately, and lets steps fail and retry in place. Failed attempts and their error observations accumulate in the transcript, exactly as they do in a real loop.

import random, statistics

SYS       = 1500        # system prompt + task, re-sent as prefill every call
OUT       = 300         # tokens the model emits per step (reasoning + tool call)
RESULT    = 500         # tool result appended on success
ERROR     = 200         # error observation appended on failure
IN_PRICE  = 3.0 / 1e6   # $/token input  (Sonnet-4.6-class pricing)
OUT_PRICE = 15.0 / 1e6  # $/token output (5x input)
N         = 8           # logical steps to finish the task
RETRY_CAP = 4           # per-step retries before giving up

def run(p, trials=200_000):
    costs, gave_up = [], 0
    for _ in range(trials):
        transcript, cost, failed = SYS, 0.0, False
        for step in range(N):
            for attempt in range(RETRY_CAP + 1):
                # one model call: prefill re-reads everything so far, plus its output
                cost += transcript * IN_PRICE + OUT * OUT_PRICE
                transcript += OUT
                if random.random() >= p:          # success
                    transcript += RESULT
                    break
                transcript += ERROR               # failure residue stays in context
                if attempt == RETRY_CAP:
                    failed = True
            if failed:
                gave_up += 1
                break
        costs.append(cost)
    return statistics.mean(costs), gave_up / trials

base, _ = run(0.0)
for p in (0.0, 0.05, 0.10, 0.20, 0.30):
    mean, gu = run(p)
    print(f"p={p:<4} ${mean*1000:6.3f}/1k runs  x{mean/base:4.2f}  gave_up={gu*100:4.1f}%")

Running it:

p=0.0  $139.200/1k runs  x1.00  gave_up= 0.0%
p=0.05 $149.476/1k runs  x1.07  gave_up= 0.0%
p=0.1  $161.370/1k runs  x1.16  gave_up= 0.0%
p=0.2  $190.392/1k runs  x1.37  gave_up= 0.2%
p=0.3  $228.183/1k runs  x1.64  gave_up= 1.9%

So 20% per-step failure costs 1.37×, not 1.20×. The overhead you actually pay (37%) is nearly double the overhead you’d naively budget (20%). That’s the first correction, and it comes entirely from the fact that retries re-send growing context and leave residue behind.

You might expect this to get much worse for long agents, since there’s more context to re-send and more residue to accumulate. It doesn’t, much:

N=5   x1.35     N=16  x1.39
N=8   x1.37     N=20  x1.40
N=12  x1.38     N=25  x1.40

In-place retry plateaus around 1.4× regardless of length. The failed attempts are localized: a bad step 12 inflates steps 13 onward, but the marginal residue is small next to the transcript that was going to be there anyway. If your agent resumes cleanly from where it failed, ~1.4× at 20% is your number, and it’s stable. Budget for it and move on.

Where the bill actually doubles

The plateau assumes something you may not have: that a failed step is resumed, not restarted. Plenty of agents can’t resume. They’re stateless between runs, or the orchestration layer’s only recovery primitive is “re-run the job,” or a failure deep in the trajectory corrupts state badly enough that starting over is the only safe option. In that world an unrecovered step failure throws away all the work before it and re-runs the whole task from step 0.

Now failures compound across the trajectory instead of staying local. Same model, restart semantics:

def run_restart(p, trials=100_000):
    costs = []
    for _ in range(trials):
        cost = 0.0
        while True:                     # keep restarting until one clean pass
            transcript, clean = SYS, True
            for step in range(N):
                cost += transcript * IN_PRICE + OUT * OUT_PRICE
                transcript += OUT
                if random.random() >= p:
                    transcript += RESULT
                else:
                    clean = False
                    break               # abort, restart from step 0
            if clean:
                break
        costs.append(cost)
    return statistics.mean(costs)

                 restart   (in-place)
p=0.05           x1.22     x1.07
p=0.10           x1.53     x1.16
p=0.15           x1.98     x1.26
p=0.20           x2.62     x1.37

At 15% per-step failure, restart semantics double the bill. At 20% they nearly triple it. The failure rate didn’t change between the two tables — the recovery architecture did. That’s the headline: your retry multiplier is set by how you recover, not by how often you fail. A 20% failure rate is a 1.4× problem if you resume and a 2.6× problem if you restart.

The mechanism is the geometric one you’d expect once you see it. The probability that an N-step run completes with no failures is (1−p)^N. At p=0.2, N=8 that’s 0.8⁸ ≈ 0.17, so you need about six full attempts on average to get one clean pass, and every aborted attempt burned real tokens on the way to failing. Longer trajectories make restart dramatically worse, exactly the opposite of the in-place case.

Retry caps, and the failure you’re hiding

Look back at the gave_up column in the first table: 0.2% at p=0.2, 1.9% at p=0.3. That’s the fraction of runs that hit the retry cap and failed for good. It’s small, and it’s a trap.

A per-step retry cap is non-negotiable — without it a persistently-failing step retries forever and a single stuck run can outspend a thousand healthy ones. But the cap converts a cost problem into a correctness problem: some runs now fail outright, and if your budget math only looks at the average bill, you won’t see them. You have to track the give-up rate as its own metric. A cap of 4 that fires 2% of the time might be fine or might be a user-facing outage depending on what the task was; the cost model can’t tell you which. It can only tell you the runs exist.

What I’d actually do

The model is a toy, but the levers it exposes are real, and they’re ordered by leverage:

  1. Resume, don’t restart. This is the single biggest factor — 1.4× versus 2.6× at the same failure rate. If your agent can’t checkpoint state and resume a failed step, that’s the first thing to build. Everything else is second order next to it.
  2. Drive down p at the worst steps, not the average. Failure isn’t uniform. One brittle tool or one ambiguous instruction usually dominates. Because cost scales with (1−p)^N under restart, halving p at the two worst steps beats shaving a point off everything. (Making tools harder to misuse is its own topic — that’s the next post.)
  3. Cap retries per step and alert on the give-up rate. The cap bounds the tail; the alert stops you from shipping silent failures as if they were savings.
  4. Budget for the multiplier you measured, not the one you assumed. Plug your real p, N, and token sizes into the model above. If you’re on restart semantics and p is anywhere near 15%, your bill is roughly double your naive estimate — know that before the invoice tells you.

The number that matters isn’t your failure rate in isolation. It’s your failure rate times your recovery architecture, and the second factor is the one you control most cheaply. Measure both before you decide retries are a rounding error.

The model in this post is a back-of-envelope Monte Carlo, not a benchmark of any specific system — the token sizes and prices are stated so you can swap in your own. The lesson (multiplicative retry cost, recovery architecture as the dominant lever) transfers across providers; the pricing ratio happens to be Anthropic’s at the time of writing.