https://loopandretry.github.io/posts/ Recent content in Posts on Loop & Retry https://loopandretry.github.io/images/og-default.png https://loopandretry.github.io/images/og-default.png Hugo en-us Thu, 16 Jul 2026 09:00:00 -0400 https://loopandretry.github.io/posts/streaming-tool-calls-without-losing-your-mind/ Thu, 16 Jul 2026 09:00:00 -0400 https://loopandretry.github.io/posts/streaming-tool-calls-without-losing-your-mind/ Streaming a text response is easy: print tokens as they arrive, order doesn’t matter to the reader. Streaming a tool call is not, because the payload is JSON, and partial JSON is not valid JSON. The three ways people handle that mismatch, why two of them break in production, and the pattern that lets you show progress without ever executing on a half-formed argument. https://loopandretry.github.io/posts/retry-budgets-by-language/ Wed, 15 Jul 2026 09:00:00 -0400 https://loopandretry.github.io/posts/retry-budgets-by-language/ A retry budget is a language-agnostic idea, but the place you enforce it is not. Python’s tenacity decorators, Go’s context-plus-backoff, and JavaScript’s promise chains each make a different mistake easy and a different guarantee hard. Where the shared budget lives, and the per-language trap that leaks it. https://loopandretry.github.io/posts/debugging-a-failed-run-costs-more/ Tue, 14 Jul 2026 13:00:00 -0400 https://loopandretry.github.io/posts/debugging-a-failed-run-costs-more/ The cheap part of a failed agent run is running it again. The expensive part is that you can’t — the failure was non-deterministic, so the run that broke is gone, and you pay to summon it back. A cost model shows why reproduction, not repair, dominates your debugging bill, and why always-on tracing is almost always cheaper than the alternative it replaces. https://loopandretry.github.io/posts/predicting-agent-failure-before-release/ Tue, 14 Jul 2026 05:02:00 -0400 https://loopandretry.github.io/posts/predicting-agent-failure-before-release/ A demo proves an agent can succeed once. It says almost nothing about how often it will fail under real load, real input distributions, and real adversarial garbage. The failures that cost you in production are predictable before release — but only if you test the things that actually shift between the demo and the deployment. Four pre-release signals that forecast production failure, and the ones that don’t. https://loopandretry.github.io/posts/how-agent-failures-cascade/ Tue, 14 Jul 2026 05:01:00 -0400 https://loopandretry.github.io/posts/how-agent-failures-cascade/ A single agent error rarely stays a single error. The bad output goes into the context, the next step reasons on top of it, and the mistake compounds down the trajectory — one wrong step becoming N wrong steps. This is the cascade, why it’s structurally different from a fleet-wide blast radius, and the three interruption points that stop a local mistake from eating the whole run. https://loopandretry.github.io/posts/cost-beyond-tokens/ Tue, 14 Jul 2026 05:00:00 -0400 https://loopandretry.github.io/posts/cost-beyond-tokens/ Everyone budgets the token bill because the provider hands you an invoice for it. But an agent in production spends across five other axes that never show up on that invoice — wall-clock latency, orchestration, tool-call fees, human review, and idle polling — and for a lot of workloads the tokens are the smallest line. A model that sums all six so you can see which one you’re actually paying. https://loopandretry.github.io/posts/cheap-first-smart-later/ Mon, 13 Jul 2026 20:30:00 -0400 https://loopandretry.github.io/posts/cheap-first-smart-later/ Most requests to your agent are easy, and you’re paying frontier-model prices for all of them anyway. A routing cascade — try the cheap model, escalate on a measurable confidence signal — cuts spend without touching output quality, if you get the escalation trigger right. Here’s the pattern, where it breaks, and the arithmetic on when it’s worth building. https://loopandretry.github.io/posts/fleet-retry-patterns/ Mon, 13 Jul 2026 18:15:00 -0400 https://loopandretry.github.io/posts/fleet-retry-patterns/ A per-step retry cap bounds a step. It never bounds a run, and it never bounds a fleet — twelve workers each retrying ‘reasonably’ is how you turn one bad deploy into a bill. The four patterns that actually put a ceiling on what a fleet of agents can spend recovering from a failure: shared retry budgets, circuit breakers, decorrelated backoff, and poison quarantine. https://loopandretry.github.io/posts/measuring-agent-failure-in-production/ Mon, 13 Jul 2026 18:10:00 -0400 https://loopandretry.github.io/posts/measuring-agent-failure-in-production/ Most agent failures don’t throw. The run returns a result, exit code zero, and the result is wrong — or it burns an hour and quietly gives up. If your monitoring only counts exceptions, you’re blind to the failures that actually cost you. A taxonomy of agent failure modes and the specific instrumentation that catches each one before your users or your bill do. https://loopandretry.github.io/posts/long-agent-runs-are-quadratic/ Mon, 13 Jul 2026 18:05:00 -0400 https://loopandretry.github.io/posts/long-agent-runs-are-quadratic/ A naive agent’s token bill doesn’t grow with the number of steps — it grows with the square of them, because every step re-reads the whole transcript that every previous step appended to. A small cost model shows the curve, and four structural moves turn the quadratic back into something close to linear without dropping information the agent actually needs. https://loopandretry.github.io/posts/your-agents-p99-is-a-different-animal/ Mon, 13 Jul 2026 18:00:00 -0400 https://loopandretry.github.io/posts/your-agents-p99-is-a-different-animal/ Average latency is the number you demo and the number nobody experiences. A multi-step agent is a sum of random variables, so its total time is dominated by the tail of each step — and the more steps you add, the more certain it becomes that at least one of them is slow. Here’s the model, why the p99 of the whole is worse than the p99 of the parts, and the two levers that actually move it. https://loopandretry.github.io/posts/compaction-is-a-lossy-operation/ Mon, 13 Jul 2026 16:00:00 -0400 https://loopandretry.github.io/posts/compaction-is-a-lossy-operation/ When the context window fills up, the standard fix is to summarize the old turns and keep going. That summary is a lossy compression step, and the thing it silently drops is usually the one early constraint the agent needs a hundred turns later. Here’s why recency-based compaction fails, a simulation of how often the load-bearing fact survives, and the rule that actually protects it. https://loopandretry.github.io/posts/tool-output-is-untrusted-input/ Mon, 13 Jul 2026 15:00:00 -0400 https://loopandretry.github.io/posts/tool-output-is-untrusted-input/ Prompt injection isn’t a prompting problem, so you can’t prompt your way out of it. It’s the same class as SQL injection: data from an untrusted source crosses into a control channel and gets executed as instructions. The web page your agent just fetched, the ticket it just read, the email in its inbox — all of it is attacker-controllable input flowing straight into the one component that can’t tell data from commands. Here’s the data-flow framing, why ‘ignore injected instructions’ can’t work, and the boundary that actually helps. https://loopandretry.github.io/posts/idempotency-keys-for-agents/ Mon, 13 Jul 2026 09:00:00 -0400 https://loopandretry.github.io/posts/idempotency-keys-for-agents/ Retrying a read is free. Retrying a write can charge a card twice, send two emails, or book two rooms — and the model has no idea it happened. Retry safety is a property you build into the tool, not a flag you set on the loop. Here’s why at-least-once delivery is the default you’re actually running, how to derive a stable idempotency key from an agent’s intent, and a dedup wrapper that makes any write safe to retry. https://loopandretry.github.io/posts/postmortem-200-dollars-retrying-a-400/ Sun, 12 Jul 2026 09:00:00 -0400 https://loopandretry.github.io/posts/postmortem-200-dollars-retrying-a-400/ An agent burned ~$200 overnight retrying an HTTP 400 — a request that was defined to fail. No component was buggy; each layer retried “reasonably.” The teardown: why retryability is a property of the error and not a default, how three nested retry caps multiply into 75 doomed attempts per item, and why per-step caps never bound a bill. With the two-line fix and a circuit breaker. https://loopandretry.github.io/posts/when-not-to-build-an-agent/ Wed, 08 Jul 2026 02:00:00 -0400 https://loopandretry.github.io/posts/when-not-to-build-an-agent/ An agent is an LLM that controls its own control flow — and that autonomy has a price you pay on every run: quadratic token cost, serial latency, and a failure surface you can’t unit-test. Most tasks people reach for an agent on are a fixed pipeline wearing a costume. Here’s the decision checklist I use, the arithmetic on what the agent tax actually costs, and the same task built both ways so you can see the difference. https://loopandretry.github.io/posts/llm-as-judge-is-lying-to-you/ Wed, 08 Jul 2026 01:30:00 -0400 https://loopandretry.github.io/posts/llm-as-judge-is-lying-to-you/ A model grading your agent’s output is the only thing that scales for subjective quality — and it’s a biased instrument you’re reading as a ruler. Here are the biases that actually move scores (position, verbosity, self-preference, leniency), why raw agreement with a human hides them, and how to validate and harden a judge with code — including why you should be reporting Cohen’s kappa, not accuracy. https://loopandretry.github.io/posts/what-to-measure-when-your-agent-works/ Tue, 07 Jul 2026 21:00:00 -0400 https://loopandretry.github.io/posts/what-to-measure-when-your-agent-works/ “It works” is a demo result, not a measurement. An agent is a trajectory, not a function, and grading only the final answer throws away most of what decides whether it’s reliable. Here’s a five-layer scheme for what to measure — outcome, trajectory, cost, failure class, and stability under nondeterminism — with a small harness that computes it. https://loopandretry.github.io/posts/context-window-is-a-cache/ Tue, 07 Jul 2026 20:00:00 -0400 https://loopandretry.github.io/posts/context-window-is-a-cache/ Treating the context window as append-only memory is how agents get slow, expensive, and quietly wrong. The fix is to run it like a cache with a budget and an eviction policy: decide what earns its tokens every turn. Here’s the cost math, the accuracy failure mode, and a working context manager. https://loopandretry.github.io/posts/loop-drift/ Tue, 07 Jul 2026 10:00:00 -0400 https://loopandretry.github.io/posts/loop-drift/ The worst agent failures don’t crash — they keep working. A postmortem on loop drift: an agent that stayed busy for 40 steps without getting closer to done, why the model’s own progress reports can’t catch it, and the external signals and evals that can.