Don't Just Trust Your Memory: Evidence-Linked Memory Reduces Memory-Induced Degradation in LLM Agents
Yukyeong Lee, Yumin Choi, Jinheon Baek, Sung Ju Hwang
ACL Rolling Review (October
cs.AI, cs.MA
2026-11-11
WikiTrace links every memory lesson to its raw trajectory so agents can check before acting: first on all 8 suites, +10.7 over no memory, regression down from 6.5% to 4.5%.
LLM agents commonly keep an external memory: trajectories of past tasks get distilled into short lessons that are injected into new tasks. The failure mode this KAIST team starts from is that memory can hurt. An agent that would solve a task blind sometimes fails with memory, because it follows a lesson that is wrong, or stretches a valid one outside its scope. The paper calls this memory-induced degradation and measures it separately from the benefit: Gain is the rate at which memory turns failures into successes, Regression the rate at which it turns successes into failures. Average accuracy hides the second number.
The diagnosis comes first. Across tasks the agent solves in all three no-memory trials but fails with memory, 52% of failures trace to a memory item the agent followed, most often an outright wrong one. The root cause is structural: a trajectory carries a single success/failure label, which never says which of its many actions caused the outcome, so any lesson-extraction method is guessing. Existing systems hand over lessons without the trajectories they came from, so the agent has no way to check. In one ALFWorld case, an ExpeL rule claims a lamp task ends once the object sits beside the lit lamp; the run that produced the rule shows the agent had to pick the object up first. The rule contradicts its own evidence, and following it fails the task.
WikiTrace keeps the evidence next to the lesson, in three linked layers:
Construction is a single pass: a compiler reads each past run (task, trajectory, binary outcome) and emits a compilation plus insights, all mounted as read-only files.
At test time the first message carries every insight plus the three most similar past runs as entry points. The prompt tells the agent to descend through the links when an insight lacks the details needed to act, conflicts with another insight, or came from a failed run, and again after any failed action. The agent then searches the linked files with read and bash for the fact that settles the question, revises the insight if needed, and acts.
The design logic: a lesson is compact and actionable but unverifiable; a trajectory is faithful but too long to reread for every decision. Linking them lets the agent pay the cost of evidence only when something looks off.
Five benchmarks, eight suites, one shared terminal agent on deepseek-v4-flash2, three trials each:
| Method | Avg over 8 suites |
| No Memory | 79.5 |
| AWM / ExpeL / ReasoningBank | 80.3 to 81.7 |
| Insights Only (same insights, no links) | 87.0 |
| WikiTrace | 90.2 |
WikiTrace ranks first on all eight suites, +10.7 over no memory. The cleanest comparison is Insights Only, which differs only in access to source runs: the 3.2-point gap is what the evidence links add. WikiTrace is also the only memory method that matches or exceeds No Memory on every suite.
On the Gain/Regression split, WikiTrace's Regression is 4.5%, against 6.5% for Insights Only and 9.7% for rerunning the agent without memory (the noise floor). It is the only method whose Regression confidence interval sits entirely below that floor, and wrong or out-of-scope items cause half as many of its regressions as Insights Only's. Its Gain exceeds the floor by 35 points, versus 27 for Insights Only and 7 to 14 for existing memory methods.
Behavior analysis: the agent descends in 22% of easy-task runs but 44% of hard ones, and descent runs beat Insights Only by 1.6 to 4.7 points more than non-descent runs, with the edge growing with difficulty. In 43% of descending runs the evidence changes the plan (revising an insight 24%, adding a missing detail 14%, supplying an uncovered fact 4%); those runs lead by 8.3 points. Test-time cost is 1.17x Insights Only. With a GLM-5.3-flash backbone the pattern repeats: 92.7 average versus 71.2 for no memory and 90.9 for Insights Only, while every existing memory method drops below no memory on at least one suite.
For teams building agents with experiential memory, the takeaway is direct: don't store conclusions only. Keeping the raw logs costs little, and the payoff is measurable both in score and in fewer broken tasks. The Gain/Regression decomposition is itself a useful evaluation recipe, since aggregate accuracy can look fine while memory quietly breaks tasks. The mechanism is also agnostic to where insights come from, so it can stack on top of ExpeL- or ReasoningBank-style memories.
Honest sizing: most of the 10.7-point jump comes from having good insights at all (Insights Only is already +7.5). The links contribute 3.2 points plus the regression cut. This is a solid systems improvement, not a new capability class.
Stated by the authors: memory is still built from binary outcomes, so a compilation's diagnosis and its insights can simply be wrong, and the source link lets the agent check a lesson without verifying it. All insights go into the first message, so the prompt grows with memory, and retrieval at scale is left open. The failure-cause and descent analyses rely on LLM judges reading the agent's reasoning, which describes what the reasoning says rather than what caused the outcome; agreement on the effect labels was weak (kappa 0.46), patched with a three-judge majority.
Three more from the setup. Baselines were normalized to a single offline build with true outcomes, which strips ExpeL of its retry-and-compare design, so existing-method numbers may be understated. The descent prompt and the mounted evidence are bundled, and no prompt-only ablation is reported. HORMA, a recent system that also connects summaries to raw history, is discussed in related work but not compared.