More Memory, Worse Agents: Error Reproduction and Anti-Persistence in LLM Agents
Ruhan Wang, Kishan Panaganti, Dongruo Zhou
cs.AI, cs.CL
2026-10-20
Anti-Persistence stores trajectories, synthesizes guidance, then drops it. Nine settings average 59.9%, vs 53.5% trajectory memory and 49.0% stored rules, at lowest LLM-call cost.
An LLM agent can improve across tasks without a weight update by rereading its own past runs. The fork in that design is what the store is allowed to keep. A trajectory record holds the task, the actions, the observations, and the outcome. Reflexion, ExpeL, and production skill libraries take a second step: they compress runs into rules, summaries, and skills, then keep the compression. Long traces mix steps worth reusing with details that matter for one episode only, so a short guide is easier to pull back in.
A bad rule does more than miss once. Once it is retrieved, later actions bend with it. Those actions are written back as experience, and the next synthesis can emit a new rule that carries the same mistake. The original text does not have to be shown to the synthesizer. The damaged trace is enough. That loop is error reproduction: abstraction, execution, then abstraction again. The mistake outlives direct lookup of the faulty rule. Deleting the parent leaves both the trajectories it shaped and the descendant rules drawn from them.
Anti-Persistence separates accumulation from abstraction. Trajectories and outcomes stay. A rule is built for the current task and thrown away when the episode ends, so the abstraction store stays empty. An outcome log records the mode, the raw success bit, and a cost-adjusted reward. It does not keep the guidance text.
Each task uses one of four modes. In none, retrieved trajectories enter the prompt and no synthesis call is made. Imitation extracts action patterns and the conditions under which they applied. Strategic mode organizes subgoals, tool use, and recovery. Corrective mode reads failures and writes avoidance rules, "do not do X, because Y," with one success as a contrast when a success exists. With no failure on hand, corrective returns empty instead of falling through to strategic, so the selector cannot credit the wrong arm.
Mode choice is a contextual bandit over those four arms. Reward is success minus λ times extra synthesis calls, clipped to [-1, 1]. Default λ is 0.05: a real synthesis call costs a little, a skipped call costs nothing. Tasks are embedded with Qwen3-Embedding-8B. The controller takes up to 10 outcome-log neighbors, decays older feedback with γ = 0.998, and adds an exploration bonus with coefficient 0.5 so rarely tried modes are not dropped. If the neighborhood is a weak match, average cosine similarity under 0.7, it falls back to the global best mode. The first three rounds rotate through all four modes. Later tasks still take a uniform random mode with probability 0.1. The rule has no confidence bound and no regret guarantee.
After the mode is fixed, retrieval pulls at most five trajectories: successes for imitation and strategy, failures for correction, plus one contrasting success on the corrective arm. The new trajectory and the mode reward are stored. The guidance text is not. Backbone, prompts, and the selection rule stay fixed. Each benchmark trains on 50 tasks a round for 10 rounds, 500 episodes. Held-out evaluation freezes memory. An episode past 50 environment steps is a failure, and evaluation logs never enter the next update.
The benchmarks are ALFWorld, WebShop, and ScienceWorld. The backbones are Llama-4-Scout, Qwen3-Coder-Next, and Gemma-4-31B-it. Held-out pools are 134, 100, and 60 tasks, three seeds. Baselines are ReAct, Reflexion, ExpeL, and AutoRefine, plus trajectory memory, abstraction memory, and joint memory.
Equal-weighted across the nine settings, Anti-Persistence reaches 59.9%. Trajectory memory is at 53.5%, abstraction memory at 49.0%, joint memory at 49.3%, and Reflexion, the strongest agent baseline on that average, at 52.9%. Anti-Persistence is first in seven of nine cells. It beats both trajectory memory and abstraction memory in every cell, and joint memory in eight. Abstractions alone, and abstractions stored beside trajectories, each lose to trajectories alone in seven cells.
| Setting | Anti-Persistence | Trajectory | Abstraction | Joint |
| Llama / ALFWorld | 54.7±2.9 | 41.6±1.7 | 42.7±1.2 | 33.3±2.3 |
| Llama / WebShop | 46.3±1.8 | 45.3±1.2 | 37.3±1.2 | 35.3±1.2 |
| Llama / ScienceWorld | 30.6±1.0 | 20.0±1.7 | 18.9±1.0 | 7.2±1.9 |
| Qwen / ALFWorld | 96.0±1.4 | 84.4±2.6 | 84.7±2.4 | 81.9±0.2 |
| Qwen / WebShop | 54.0±2.0 | 50.7±1.2 | 38.3±3.5 | 47.0±1.7 |
| Qwen / ScienceWorld | 52.6±1.5 | 43.8±3.7 | 34.0±1.0 | 36.3±0.6 |
| Gemma / ALFWorld | 97.3±1.2 | 96.0±1.0 | 92.7±2.3 | 95.3±3.1 |
| Gemma / WebShop | 53.3±3.1 | 48.0±0.2 | 45.3±4.2 | 51.3±1.2 |
| Gemma / ScienceWorld | 54.1±6.3 | 51.7±1.7 | 46.7±1.7 | 56.3±2.4 |
The weak backbone is where the gap opens. On Llama, ALFWorld moves from 41.6% with trajectory memory to 54.7%, and ScienceWorld from 20.0% to 30.6%, while joint memory falls to 7.2%. On a strong model, ALFWorld is nearly full: Gemma Reflexion scores 98.0% and Anti-Persistence 97.3%. On Gemma ScienceWorld, joint memory is ahead, 56.3% to 54.1%.
Anti-Persistence also has the lowest average LLM-call count of the four memory designs, counting synthesis plus execution. Cells do not all agree. On Llama ScienceWorld, abstraction memory uses 1.33 calls and joint memory 1.15, against 3.56 for Anti-Persistence, with success at only 18.9% and 7.2%. Short traces line up with quitting early, and the cause is not isolated. On Qwen ALFWorld, trajectory memory uses 10.90 calls and Anti-Persistence 11.84, with success 84.4% versus 96.0%.
Keeping versus discarding the same synthesized text is reported in four settings. Discarding wins all four. Llama ALFWorld moves from 26.7% persisted to 54.7% discarded, up 28.0 points, and WebShop gains 12.3. Qwen gains 3.3 and 6.7. A weak model pays more for storing a bad rule.
On Qwen the bandit scores 96.0% and 54.0%, above fixed modes and random choice. Strategy is the best fixed policy on ALFWorld, imitation on WebShop. γ = 0.998 is the best value tested. At 0.980 success is 90.0% and 48.7%; at 0.990 it is 87.3% and 44.7%; with no decay, 88.7% and 48.7%. The curve is not monotone. λ = 0.05 is best as well. Removing the penalty, or raising it to 0.20 or 0.50, lowers success.
AutoRefine keeps skills and drops the bottom 20% by utility every two rounds. Across the six Llama and Qwen cells it beats abstraction memory once, ties once, and loses four times, and it trails Anti-Persistence in all six. Retrieval and synthesis change too, so this is not a pure pruning test. Anti-Persistence leads those six cells at round 10, not at every earlier round. Llama joint memory slides late, most clearly on ScienceWorld.
Skill libraries assume experience should be distilled and then kept. The nine-setting average goes the other way. Stored abstractions trail raw trajectories, and keeping both trails trajectories too. What holds up is storing the runs and building the rule for this task only.
On-demand synthesis sounds expensive. Averaged over the nine settings it is the cheapest of the four memory designs, because none skips the extra call and a 0.05 penalty keeps synthesis uncommon. The gain is large on the weak backbone: discarding rules instead of storing them is worth 12.3 and 28.0 points on Llama. A system that already persists skills can run that one comparison before adding another maintenance pass. Selective retention after an actual check is left open. The backbone and the prompts stay frozen, and no weights move.
The indirect channel is still open. Dropped guidance can shape the trajectory that gets stored, and the next synthesis can read that trajectory and write the error again. An empty abstraction store blocks direct reuse only. Temporary guidance can still bias later traces and later mode choices.
The evidence for error reproduction is an association between earlier abstractions and later ones. Labels track downstream success. They do not identify the causal effect of any one rule. The formal split is exposure, flagged generation, and admission into memory. A positive probability is not a demonstration that errors grow every round. The Figure 1 caption says wider spread is not guaranteed.
Averages hide cells that point the other way. Joint memory beats Anti-Persistence on Gemma ScienceWorld, 56.3% to 54.1%, and Reflexion is slightly ahead on Gemma ALFWorld, 98.0% to 97.3%. WebShop is scored on 100 instructions, not the benchmark's 500-instruction test split. ScienceWorld uses five training instances and two held-out instances of each task type, a different split from prior work. Three seeds leave the Gemma ScienceWorld cell at ±6.3.
The bandit has no regret guarantee, and γ is brittle: leaving 0.998 makes success worse, and not in a monotone way. The 0.7 similarity fallback comes from a small validation set. Mode shares are ten-round averages and say nothing about convergence. The AutoRefine reimplementation uses one contrastive synthesis call rather than the original two-call procedure, so pruning is not isolated. The headline table covers household tasks, shopping, and elementary science, with train and held-out tasks drawn from the same families.