Tokens on Loop & Retry https://loopandretry.github.io/tags/tokens/ Recent content in Tokens on Loop & Retry Loop & Retry https://loopandretry.github.io/images/og-default.png https://loopandretry.github.io/images/og-default.png Hugo en-us Tue, 14 Jul 2026 05:00:00 -0400 Your token bill is the cheap part: dimensioning the real cost of an agent https://loopandretry.github.io/posts/cost-beyond-tokens/ Tue, 14 Jul 2026 05:00:00 -0400 https://loopandretry.github.io/posts/cost-beyond-tokens/ Everyone budgets the token bill because the provider hands you an invoice for it. But an agent in production spends across five other axes that never show up on that invoice — wall-clock latency, orchestration, tool-call fees, human review, and idle polling — and for a lot of workloads the tokens are the smallest line. A model that sums all six so you can see which one you’re actually paying. Why a long agent run costs O(N²) tokens — and how to flatten it https://loopandretry.github.io/posts/long-agent-runs-are-quadratic/ Mon, 13 Jul 2026 18:05:00 -0400 https://loopandretry.github.io/posts/long-agent-runs-are-quadratic/ A naive agent’s token bill doesn’t grow with the number of steps — it grows with the square of them, because every step re-reads the whole transcript that every previous step appended to. A small cost model shows the curve, and four structural moves turn the quadratic back into something close to linear without dropping information the agent actually needs. The context window is a cache, not a memory https://loopandretry.github.io/posts/context-window-is-a-cache/ Tue, 07 Jul 2026 20:00:00 -0400 https://loopandretry.github.io/posts/context-window-is-a-cache/ Treating the context window as append-only memory is how agents get slow, expensive, and quietly wrong. The fix is to run it like a cache with a budget and an eviction policy: decide what earns its tokens every turn. Here’s the cost math, the accuracy failure mode, and a working context manager. Retry budgets: why 20% per-step failure doubles your token bill https://loopandretry.github.io/posts/retry-budgets/ Sun, 05 Jul 2026 09:00:00 -0400 https://loopandretry.github.io/posts/retry-budgets/ Retries feel cheap and local. In a multi-step agent they’re neither. A small cost model shows why 20% per-step failure can more than double your bill — and how your recovery architecture, not your failure rate, decides the multiplier.