Agents Need to Forget on Purpose

The newest agent papers sharpened a problem I have been circling for months: persistence is easy to celebrate and hard to govern. An agent can save memories, accumulate skills, reuse cached context, and stretch a task across hundreds of steps. None of those capabilities guarantees that the information is still valid, the skill still helps, the retrieved context deserves its latency budget, or the agent is still following the user's actual policy.

I used to think of memory quality mostly as a retrieval problem. Now I think the harder question is lifecycle control. A useful agent needs to know what can remain active, what should be compressed, what must be revoked, and what evidence would justify bringing it back.

Persistence creates falsifiability debt

TEPA starts from a concrete failure mode called memory pollution. When the world changes, an append-only memory can keep retrieving old facts beside new ones. More memory then makes the agent worse because superseded evidence still occupies the active prompt.

The paper makes validity an explicit state. Observations are stored as keyed precedents, and fresh contradictory evidence can revoke an active precedent under the same key. The old record remains available for audit, but it is no longer treated as current evidence.

That separation matters. Deleting history makes debugging difficult, while leaving every historical claim active makes retrieval unreliable. Revocation gives the system both a current view and an explanation of how that view changed.

The reported controlled-drift result is striking: TEPA scored 0.950 during full reversal, while append-only and last-write-wins memory both fell to 0.210 and no memory scored 0.309. Those numbers come from the authors' preprint and should not be generalized to every memory stack, but the failure pattern is useful. Persistence without a validity operation can perform worse than forgetting everything.

This maps directly to how I want project memory to work. Canonical files should remain the source of truth. Retrieved notes should carry a source, timestamp, scope, and active or revoked state. When a resume date, deployment status, or repository decision changes, the old evidence should remain inspectable without continuing to steer the next run.

Skills need reversible optimization

Memory is not the only thing that accumulates technical debt. Agent skills can also grow into long prompt files full of overlapping advice, outdated commands, and rules that helped one task but hurt another.

SkillProx treats skills as reusable textual artifacts and gives their evolution a more explicit loop. Its forward stage applies diagnosis-driven edits, measures the result on the same task batch, and rolls back regressions. Its backward stage decomposes the skill into auditable knowledge units, estimates their utility with a leave-one-out test, and then consolidates, demotes, or removes them behind validation gates.

I like the emphasis on deletion as its own operation. An edit instruction such as "make this skill better" has no natural pressure toward simplicity. It can keep adding caveats forever. A dedicated removal step asks a sharper question: if this unit disappears, does performance actually decline?

The authors report a 3.0 percentage point average accuracy gain over their strongest gradient-based baseline across several models and in-distribution and out-of-distribution benchmarks. The exact gain will depend on the tasks, but the mechanism is more interesting than the headline number. A maintained skill should have a diff, an evaluation, a rollback path, and a reason each rule survived.

Installing a capability has a carrying cost

The usual advice to avoid reinventing the wheel feels incomplete to me. A README keyword can convince a developer to add a library without reading the code path, defaults, failure behavior, or permissions that the application will depend on. The dependency shortens the first patch while leaving critical behavior unexplained.

A small replacement carries its own maintenance and correctness risk. I want the choice to start with the existing implementation. A focused implementation can make sense when it is easier to inspect and test than a broad dependency. A mature library can make sense when its boundary already matches the problem. If I cannot explain which code path handles my input and how it fails, I have not finished evaluating the dependency.

I notice a related habit in my personal agent setup. Installing an MCP server, copying a skill into Codex or Claude Code, or registering another agent takes seconds because the capability may become useful later. The extension can remain installed for months without serving a real task.

An unused capability can have a cost before any tool runs. Clients that expose all installed capabilities can place skill catalogs, routing instructions, tool names, and JSON schemas into the model's available context. That material competes with the repository, task, and evidence the model needs now. A larger tool surface also gives the model more irrelevant choices and expands the permissions and dependencies a developer has to audit. Other clients load definitions on demand, so I would measure the token and latency cost for the client instead of assuming every installation behaves the same way.

I want my agent setup to work more like a maintained dependency graph than a collection. Each MCP server or skill should have a source, a reason for installation, a last-used record, a clear permission boundary, and a simple way to disable it. Project-specific capabilities should stay with the project when possible. If removing a capability does not hurt a representative task, it should leave the default prompt and routing surface.

Retrieval should spend attention like a budget

Even valid memory can be too expensive to load. Long-context RAG often retrieves coarse chunks, then spends prefill time processing repeated or irrelevant text.

CoinRAG moves cache reuse below the chunk level. It identifies smaller query-relevant semantic units, reuses their precomputed KV representations, and composes them with chunk-level context through a two-stage retrieval process. On LongBench multi-hop question answering, the authors report a new latency and accuracy Pareto frontier with an average 5.3 percent relative F1 improvement under a fast prefill budget.

The product lesson is not that every application should implement sliced KV caches. It is that retrieval has at least two budgets: evidence quality and attention cost. A result can be relevant enough to rank highly but still contain far more tokens than the next action needs.

For a coding agent, I would want the retrieval record to show why a file or snippet was selected, how much context it consumed, and whether a smaller evidence unit would have supported the same decision. That makes compression measurable instead of cosmetic.

Long tasks need feedback before the end

The Horizon Gap surveys 1,547 papers and separates three ideas that are often blended together: long-horizon tasks, long-context models, and long-term memory systems. A larger context window does not automatically make a multi-hour task reliable, and a persistent memory store does not ensure that the execution remains on course.

The survey's recurring observation is that outcome-only signals become less informative as task horizons grow. By the time a long run fails, the final score may say very little about which planning decision, retrieval, tool call, or premature completion claim caused the failure. Long-horizon systems need denser process and trajectory signals.

LivePlan gives that idea a practical shape for programming agents. A deterministic monitor watches the trajectory for drift and inefficiency, and it calls an advisor model only when a rule detects a problem. The paper reports an average 9.9 percent improvement in issue resolution over vanilla SWE-agent at an additional average cost of $0.08 per instance.

The architecture is appealing because it does not ask another model to narrate every step. Cheap checks handle observable failures such as repetition, plan drift, or suspicious termination. Model judgment is reserved for moments that need interpretation.

That is close to the engineering loop I trust: establish a plan, inspect the actual state, run mechanical gates, and intervene when the evidence stops matching the claim.

Completion is not the same as fidelity

WebRider studies a related gap in live-web assistance. Its audit found that a strong controller completed 99.2 percent of tasks but honored every delegated policy constraint in only 38.8 percent of cases. The agent often reached a plausible result without preserving the user's requirements for evidence, uncertainty, preferences, or stopping behavior.

WebRider represents those requirements as an intent contract that remains visible across the browsing path. The contract records goals, constraints, evidence obligations, answer form, and task-local persona controls. Evaluation can then grade both the result and whether the route to that result preserved the delegated policy.

This is especially important for agents that can mutate external systems. An agent that opens the right page by using an unapproved account session did not succeed. An agent that produces a correct code patch but skips the required verification did not fully complete the task. The operating path is part of the output.

What I would build next

If I turned these papers into one portfolio project, I would build a lifecycle layer for agent memory and skills.

1. Every memory would store its source, key, timestamp, scope, confidence, and validity state.
2. Contradictory evidence would revoke an active memory while preserving the old record for audit and possible re-promotion.
3. Skills would be split into testable units with measured utility, validation-gated edits, automatic rollback, and last-used records for installed capabilities.
4. MCP servers and agent tools would report their prompt footprint and remain disabled outside the projects that need them.
5. Retrieval would log both evidence relevance and context cost, making it possible to compare chunks against smaller semantic units.
6. Long runs would have deterministic trajectory checks for repetition, drift, unsupported completion, and missing verification.
7. The final review would grade policy fidelity as well as task completion.

The interface could stay deliberately small. I would rather have a precise table of active and revoked evidence, a readable skill diff, and a replayable trace than another animated agent graph that hides the important state.

My takeaway

The next useful memory feature may be a revoke button, not a larger context window.

Reliable agents need persistence, but persistence has to remain falsifiable. Memories need validity states. Skills need pruning and rollback. Installed tools need a reason to remain active. Retrieval needs an attention budget. Long tasks need process signals. Delegated work needs a contract that survives the path to completion.

I still want agents that learn from experience. I just do not want yesterday's experience, or a tool I installed once and never used, to become permanent context for every task that follows.