Every long-running agent eventually hits the wall: the context window fills, and something has to give. The near-universal fix is compaction — summarize the older turns into a shorter recap, drop the raw transcript, and keep going. It works, right up until the run where the agent cheerfully violates a constraint it was given on turn 3, because turn 3 didn’t make it into the summary. Nothing errored. The agent just forgot, and forgot in a way that looks exactly like a reasoning failure instead of the data-loss bug it actually is.

This post is about treating compaction as what it is: a lossy compression step in the middle of your control flow. I’ve argued before that the context window is a cache, not a memory — that you should run it with a budget and an eviction policy. Compaction is that eviction policy, executed by a model that doesn’t know which facts are load-bearing. That’s the problem.

Why “summarize the old turns” loses the wrong thing

Compaction usually works on recency. The last few turns stay verbatim; everything older gets squeezed into a paragraph or two of summary. This is a reasonable default for the shape of a conversation — recent context is usually most relevant to the next step — and it is exactly wrong for a specific, common, and costly case.

The facts that matter longest are often stated earliest. The user’s hard constraint (“never touch the production database”, “the budget is $500, hard cap”, “the customer is in the EU so GDPR applies”) arrives at the start of the task and stays relevant until the end. A recency-based summarizer sees that fact age out of the verbatim window, tries to compress it alongside fifty other turns of tool calls and chit-chat, and — because a summary’s whole job is to drop detail — renders it as “the user described some requirements” or drops it entirely. The load-bearing constraint gets the same treatment as the small talk.

The failure is delayed and disguised. The agent runs fine for eighty turns. Then it reaches the step where the dropped constraint would have applied, doesn’t have it, and does the reasonable-looking wrong thing. You debug it as a reasoning error or a prompt problem. It’s neither. It’s a fact that was in context, got compressed out, and never came back.

Simulating how often the fact survives

Let’s put numbers on it. Model a run as a stream of facts arriving over turns. Most are ordinary (tool results, intermediate reasoning). One, planted early, is load-bearing: it’s needed at the very end. When the transcript exceeds a budget, we compact. Two policies:

  • Recency: keep the most recent facts that fit; summarize the rest into a lossy blob that retains each old fact only with probability RETAIN (a summary keeps some things, drops others).
  • Salience: same, but facts tagged load-bearing are pinned — never summarized away.

We measure one thing: at the final turn, is the load-bearing fact still present?

import random

TURNS      = 120      # length of the run
BUDGET     = 30       # facts we can hold verbatim before compacting
RETAIN     = 0.30     # chance the running summary keeps a given old fact
KEY_TURN   = 3        # the load-bearing constraint arrives early

def simulate(policy, trials=200_000):
    survived = 0
    for _ in range(trials):
        kept = []            # facts still held verbatim (True = load-bearing)
        key_state = None     # None=still verbatim, True=in summary, False=dropped
        for t in range(TURNS):
            kept.append(t == KEY_TURN)
            if len(kept) > BUDGET:
                # compact: oldest facts leave the verbatim window into the summary
                old, kept = kept[:-BUDGET], kept[-BUDGET:]
                for is_key in old:
                    if not is_key:
                        continue
                    if policy == "salience":
                        key_state = True                     # pinned: always retained
                    elif key_state is None:                  # decided once, then carried
                        key_state = random.random() < RETAIN
        present = any(kept) or key_state is True
        survived += present
    return survived / trials

for policy in ("recency", "salience"):
    print(f"{policy:9} load-bearing fact present at end: {simulate(policy)*100:5.1f}%")

Running it:

recency   load-bearing fact present at end:  30.0%
salience  load-bearing fact present at end: 100.0%

Under recency-based compaction, the fact the whole task depends on is gone 70% of the time by the end of a long run. Not because the window was too small to hold it — it’s one fact — but because the compaction policy had no idea it was special. It aged out, got summarized, and the summary rolled the dice and lost. The salience policy keeps it every time, at the cost of pinning one fact.

The exact percentage isn’t the point (it’s set by RETAIN, which you can argue about). The point is the shape: once an early critical fact ages out of the verbatim window, its survival collapses to whatever your summarizer’s retention rate happens to be — here 30% — and no additional run length ever recovers it. The longer the agent works, the further behind that fact falls, and the more certain it is to have forgotten why it started.

The rule: compaction needs a schema, not just a summarizer

The fix isn’t a better summarization prompt. A better prompt still asks one model to guess, in one shot, which of a hundred turns will matter a hundred turns from now — and it will still sometimes guess wrong, silently. The fix is to stop treating all context as fungible text to be compressed uniformly, and give compaction a schema of what must never be dropped.

Concretely, before you compact, separate context into two bins:

  1. Durable state — pinned, never summarized. Constraints, hard limits, IDs and keys, the task goal, decisions already made, and anything the user flagged as important. This is small and it is load-bearing. It should live in a structured slot that compaction physically cannot touch — not buried in the transcript hoping a summarizer keeps it. If you can’t enumerate this bin for your agent, that’s the actual gap: you don’t yet know what your agent must remember.
  2. Transient context — free to compress. Tool call chatter, intermediate reasoning, superseded attempts, resolved sub-tasks. Summarize this aggressively; it’s genuinely recency-biased and losing detail here is fine.

The mechanism that makes this work is the same one from the cache post: an eviction policy that knows the cost of a miss. A cache that evicts your session token because it hasn’t been read in a while is broken; so is a compactor that summarizes away the one constraint the task hinges on. Both are eviction without regard to what a miss costs. Pinning durable state is just declaring, up front, which misses are unaffordable.

Two guardrails I’d add on top: make the durable bin auditable — log what’s pinned, so when the agent does something that violates a constraint you can immediately check whether the constraint was even present (turning a mystery reasoning bug into a one-line data-loss check); and verify after compaction, not just before — a cheap assertion that every pinned fact is still retrievable in the compacted context catches a broken compactor the same run it breaks, instead of eighty turns later.

Compaction is not a neutral housekeeping step. It’s a lossy write in the middle of your agent’s memory, performed by a component that doesn’t know what’s load-bearing unless you tell it. Tell it. The alternative is an agent that runs beautifully for a hundred turns and then forgets the one thing you gave it first.

The simulation here is a toy model of fact survival, not a benchmark of any specific compaction implementation — RETAIN and the run length are stated so you can plug in numbers that match yours. The lesson (recency compaction degrades toward its retention rate for early critical facts, pinning durable state fixes it) is architecture-level and provider-independent.