Kini o ṣẹlẹ
A new preprint posted August 11 names an observed failure mode in agentic coding repositories "catastrophic remembering": project instruction files keep accumulating rules because adding a precaution is cheap, while safely deleting an old rule becomes harder after its original rationale disappears.
The researchers assembled 247,694 instruction lifetimes and 299,440 commit-to-commit transitions from 1,867 public repositories containing files such as CLAUDE.md. They tracked individual directives through edits and report that the files grew by 226% over their observed lifetimes, adding an average of 4.9 net instructions per modifying commit. The study treats those figures as evidence of persistent accumulation in its sample, not proof that every instruction was unnecessary or that all agentic coding projects behave the same way.
The paper's central mechanism is asymmetric evidence. A maintainer or coding agent can append a new instruction after a mistake without reconstructing every interaction among the rules already present. Later deletion is riskier: once the motivating failure and surrounding context have vanished, removing one directive may require checking whether it still prevents a regression under many combinations of the remaining directives. In the repository data, the estimated deletion hazard fell as an instruction aged, with a reported log-hazard slope of -0.032 per commit.
To test a possible remedy, the authors created instruction-following tasks by inverting IFEval constraints, then compared prompts containing bare rules with prompts that also preserved short comments explaining why each rule existed. In their controlled setup, uncommented prompts accumulated 211.3% excess instructions, whereas commented prompts ended at 1.4% excess. The comments were not extra commands for the model; they were compact provenance intended to make later removal decisions auditable.
The team also evaluated whether smaller, better-documented prompts helped agents follow instructions. On its WildIFEval-derived , the paper reports gains of as much as 23.1 percentage points when rationales were preserved and obsolete rules could be removed. Those results are claims from a newly posted, non-peer-reviewed preprint. The work does not establish that comments alone will improve every coding agent, repository, language, or production workflow.
Awọn alaye orisun: Catastrophic remembering research paper on arXiv ↗
Kini idi ti o ṣe pataki
Repository-level instruction files are becoming durable operational memory for coding agents, so uncontrolled growth can raise token costs, preserve obsolete constraints, and make the rules governing automated code changes harder for people to understand.
The practical risk is not simply a long file. Every instruction competes for a model's attention and can interact with newer rules, tool descriptions, code context, and the user's request. A directive written to prevent one historical failure may become irrelevant after the codebase changes, conflict with a newer policy, or overconstrain unrelated work. If nobody can reconstruct why it exists, the safest local choice is often to retain it, which moves cleanup costs into future commits.
That pattern matters beyond CLAUDE.md. Teams increasingly store conventions, safety boundaries, test commands, architectural decisions, and deployment cautions in machine-readable project guidance. These files can improve consistency and reduce repeated mistakes, but they also become a governance surface: people should be able to identify who introduced a consequential rule, what evidence justified it, and what condition would allow it to be retired. A rationale comment is a lightweight version of that audit trail.
The result suggests a useful design principle for agent memory: remembering should include the conditions for forgetting. Instead of recording only "always do X," a system can preserve the observed failure, the scope of the rule, the relevant component or test, and a review trigger. That does not automate deletion, but it gives a later maintainer evidence for deciding whether the constraint still protects correctness or merely reflects an old environment.
There is also a public-interest angle as coding agents touch more consequential software. Accumulated private instructions can quietly shape security decisions, accessibility behavior, data handling, or how an agent responds to failures. Smaller files are not automatically safer, and aggressive cleanup could remove a vital safeguard. The useful outcome is traceability: fewer unexplained rules, explicit ownership, and review practices that let humans challenge both additions and deletions.
Ibaraẹnisọrọ Mechanism: Bii O Ṣe Nṣiṣẹ Lootọ
Ṣawari imọ-ẹrọ abẹlẹ lẹhin idagbasoke yii ni ibaraenisọrọ.
crm_get_transaction(id='4092').What most distinguishes an AI agent from a basic chatbot?
Kini lati wo tókàn
The next test is independent replication across languages, organizations, agent products, and longer-lived repositories, followed by prospective trials that measure whether documented cleanup improves code quality without deleting important safeguards.
The observational sample has selection limits. Public repositories that commit agent instruction files may differ from private enterprise codebases, and repository history cannot reveal every off-platform discussion or incident that motivated a rule. Growth can also be rational when a project expands. Future analyses should separate useful coverage of new components from duplicate, contradictory, or obsolete instructions and report how results vary by repository age, size, language, contributor count, and agent platform.
The controlled experiments need broader validation. The tasks were derived from instruction-following benchmarks rather than months of live software maintenance, and the paper's matching pipeline was checked on a 50-transition sample. One author produced the hand used in part of the validation. The study did not cover non-English instruction files, and it did not sweep every threshold used to decide when a change counted as a rewrite rather than a continuation of an instruction.
Comments can preserve incorrect rationales as easily as correct ones. Teams should therefore test structured provenance against alternatives such as linked issues, failing regression tests, expiration dates, ownership fields, or automated checks that flag duplicate and conflicting directives. The safest workflow would propose removals, show the evidence and affected tests, and require review for high-impact rules rather than allowing an agent to prune instructions solely to save tokens.
Useful follow-up evidence would measure end-to-end outcomes: prompt size, instruction compliance, task success, regressions, review time, and the number of rules restored after deletion. Researchers should also test whether models actually use rationale comments as intended or sometimes mistake them for additional requirements. Until those results arrive, catastrophic remembering is a well-supported description of the authors' dataset and experiments, not a universal law or a reason to erase mature project guidance wholesale.