Memory-Centric Continual Learning for AI Agents
Dongfang Li, Zhuoen Chen, Bonian Jia, Zixuan Liu, Zhengxuan Lu, Can Deng, Yukun Shi, Yuhang Meng, Xin Zhang, Jiahe Huang, Fuhao Li, Junmai Wang, Shunyao Wen, Jingjun Sun, Jiaxu Tian, Jiangling Zhang, Zihan Wang, Liangyou Li, Yitong Yao, Shuangyong Song, Beilun Wang, Pengxu Wei, Hanlin Xu, Chang Liu, Yongxiang Li, Baotian Hu, Xiangyang Ji, Min Zhang
cs.CL, cs.AI
2026-10-21
MCCL treats agent memory as two loops: update state inside, revise policy from feedback outside, score the stateful-minus-reset gap, not recall. Scaling plots are synthetic.
Agents that can plan and call tools still repeat the same mistakes over a long deployment. The weights stay frozen, and the context window is capped by the session. Stuffing logs back into the prompt dilutes the useful signal and adds latency as trajectories grow. Continual fine-tuning can change behavior. It also costs compute, risks catastrophic forgetting, and is hard to roll back.
External memories in the MemGPT and Mem0 line are usually scored by whether an old fact comes back. A successful lookup is a different claim from a better next action. As the store grows, redundant and conflicting entries raise retrieval noise, and a shift in the task distribution produces negative transfer. Memory-Centric Continual Learning (MCCL) leaves backbone gradients alone and treats memory as the learning system: write experience, resolve conflicts, and change later actions.
Three axes separate choices that usually share one label. The substrate is where state sits. The asset is what a retrieved item is for. The policy decides writes and deletions. Owning an external store does not transfer a skill.
There are four substrates. Context is the tokens currently in view. It needs almost no architecture change and loses stability as the sequence grows. Summaries extend the horizon; the cut is lossy and irreversible. Activation memory is a hidden state or a KV cache. It joins the forward pass but is tied to one model. Moving the cache to disk does not make it an external text store, because only that model can read it. External memory is logs, vectors, or graphs. Capacity grows apart from model size, and a single record can be edited. When its rank drops, the record is still there and the behavior is gone. That is functional forgetting. Parametric memory is weights or adapters updated after deployment. A bad feedback signal can be baked into the weights.
Assets are typed by use, not by where the bytes sit. Episodes keep the event. Semantic items are facts with the situation stripped off, which helps multi-hop reuse and also turns a one-off correlation into a general claim. Procedures are workflows or code and constrain the next action. The same passage is semantic if it is read as a fact and procedural if it is read as an instruction. Consolidation compiles repeated regularities into skills. The further that skill sits from the original evidence, the cheaper it is to invoke, and the wider a bad generalization hits.
Policies can be stacked. Prescribed control is a fixed rule such as top-k. What counts as important is not revised after outcomes arrive. Deliberative control asks the LLM case by case, and a later score is not credited back to that decision. Learned control updates writes and deletes from reward. Most of those controllers are trained offline and then frozen. Online policy evolution would keep changing the rule during deployment. Long-delay feedback still does not assign credit to a specific earlier step.
The inner loop writes the new experience into state under the current policy. The outer loop revises that policy from later task feedback. The update rule can be a no-op, in which case only the contents change. A changed store counts as learning only if a later step uses it and behavior changes. Writing too freely pollutes memory: a hallucination is retrieved again and again. Provenance omitted at write time cannot be rebuilt by consolidation. Repeated LLM rewriting raises utility and then lowers it. New evidence should not automatically overwrite the old record, and skills compiled from that record have to be revised with it. A delete operation does not mean the system forgets the right things.
Feedback enters at three levels. Reflexion and Voyager only enrich the store, with verbal post-mortems or skills admitted after execution checks. AgeMem and UMA train a controller and freeze it. MemRL replaces pure similarity with a utility based on whether the memory historically contributed to success. Continual revision of the management rule during deployment is the level current systems do not close.
There is no new end-to-end run, and no accuracy gap between MCCL and MemGPT, Mem0, or a long context. Figure 1 is labeled as a synthetic illustration of allocation and interference, not a scaling exponent fit on real tasks.
Three behaviors are in scope. Retention: after delay and interference, the agent still beats a matched run with that experience removed. Adaptation: after facts, preferences, or tools change, behavior follows, and unchanged probes are not damaged. Transfer is split into near-duplicates, unseen variants of the same family, and shared structure on a different task, with negative transfer reported on its own. Cited prior work says raw trajectories hurt cases a no-memory agent already failed, while skill libraries are steadier on clear workflows and can be matched by ordinary context accumulation on open tasks. No percentages are given.
| Metric layer | What it shows | What it does not show |
| Content fidelity | Whether observations were stored intact | Whether they are later used |
| Retrieval access | Whether the item can be found | Whether it is ignored or misapplied |
| Single-task score | Whether the current probe rose | Whether the gain survives interference |
A four-gate cascade localizes the break: preservation, access, utilization, and net effect. Saying the system remembers stops at access. More capacity can raise preservation and worsen access, because retrieval gets more crowded. Two cited failure modes have the same shape. With a fixed backbone, inserting and evicting in an external store lowers earlier tasks. Extracting experience and injecting it again can turn a one-round gain into a progressive collapse. Magnitudes are not reported.
Delta Pt is the stateful score minus the reset score on the same task. No-memory, full-history, static RAG, and matched-compute baselines rule out extra prompt text and extra inference. If conversation history, tool state, and the external store move together, a positive gap does not say which piece caused it. A retention ratio is reported only when the immediate gain is above zero, so a failure to learn is not counted as a failure to remember. The paper gives no numeric Delta Pt on any task.
| Benchmark | Claim the protocol can support |
| LoCoMo, LongMemEval, PerLTQA, MemoryBank | Recall over a static history |
| MemBench | Access efficiency as scale changes |
| MemoryAgentBench | Conflict handling as facts change |
| CL-Bench, LifelongAgentBench, AgentCL | A stateful-versus-reset gap when tasks give feedback |
Attainable error, epsilon-star, depends on accumulated experience, capacity, read budget, and write budget, with the backbone and the task distribution held fixed. The prediction is a run of resource-limited regimes: error falls as relevant experience grows, then saturates when capacity or read-write compute binds. Non-parametric and parametric memory share that frontier. They differ in how a new write interferes with old content. In the illustration, editing the backbone calls for a low capacity share and a dedicated module calls for a high one. Scores cannot be compared across papers. Context budgets and inference compute are rarely matched.
The piece a team can use is the protocol. Figure 5 says its cells are design choices, not a performance ranking. Figure 3 is the same kind of map for substrates, assets, and policies.
A recall number or a single-turn score stays at the access layer. A budget-matched Delta Pt, measured again after interference, separates learning from a longer context. Growing the store alone hits retrieval noise first. The online policy loop is not closed. Recursive self-improvement, meaning a change in how later experience is organized, stays a definition. Credit assignment is still open.
The value of a write or a delete often shows up several steps later, and a final score does not identify which memory decision caused it. Consolidation that drops time, participants, and session scope lets a compressed passage be reused as evidence. Routing, which store is authoritative under conflict, and where a bad update rolls back are left open for hybrid substrates.
The two loops are definitions. There is no released implementation, no ablation, and no Delta Pt on a shared backbone. The scaling curves are illustrations. The axes are also less tidy than the names. MemGPT appears under both prescribed and deliberative control, and one record can be an episode, a fact, and a skill at once. The taxonomy can take a system apart. It does not predict which combination wins on which benchmark. Static-history suites measure recall. Sequential environments with feedback are closer to continual learning, and their budgets still do not line up, so the published scores cannot be ranked against each other.