Eviction and summarization keep a long-running agent’s window small, but the summaries themselves still accumulate in the linear transcript — and on a run long enough, that accumulation becomes the new quadratic. Past a few hundred steps, the fix isn’t a better compaction policy inside the window. It’s moving history out of the window entirely and querying it like a database instead of replaying it like a log.
Prompt caching can cut your input-token bill by 90% on the reused part of a request — or do nothing at all, with no error to tell you which. It’s an exact-prefix match with a short TTL, and small, common mistakes in how agents build requests break it silently. Here’s what actually gets cached, the five ways real agents lose the discount without noticing, and how to check whether yours is.
The default retry pattern — catch the failure, append ’that didn’t work, try again,’ resend the same messages — doesn’t give the model a clean second attempt. It gives the model a context window containing its own wrong answer, which is exactly the thing most likely to make the second attempt rhyme with the first. Why retries poison the window, how to tell a poisoning retry from a safe one, and a scrub step that keeps the constraints without keeping the wrong path.
Fine-tuning example length is a design decision, not a byproduct of your data. Pad too short and you teach a distribution you’ll never see at inference; let examples sprawl and you pay quadratic training cost to memorize context you should be retrieving. How to size training sequences to the context you’ll actually serve.
A single agent error rarely stays a single error. The bad output goes into the context, the next step reasons on top of it, and the mistake compounds down the trajectory — one wrong step becoming N wrong steps. This is the cascade, why it’s structurally different from a fleet-wide blast radius, and the three interruption points that stop a local mistake from eating the whole run.
A naive agent’s token bill doesn’t grow with the number of steps — it grows with the square of them, because every step re-reads the whole transcript that every previous step appended to. A small cost model shows the curve, and four structural moves turn the quadratic back into something close to linear without dropping information the agent actually needs.
When the context window fills up, the standard fix is to summarize the old turns and keep going. That summary is a lossy compression step, and the thing it silently drops is usually the one early constraint the agent needs a hundred turns later. Here’s why recency-based compaction fails, a simulation of how often the load-bearing fact survives, and the rule that actually protects it.
Treating the context window as append-only memory is how agents get slow, expensive, and quietly wrong. The fix is to run it like a cache with a budget and an eviction policy: decide what earns its tokens every turn. Here’s the cost math, the accuracy failure mode, and a working context manager.