Can LLM Agents Organize Their Own Memory?
Zichuan Fu, Junjia Qi, Wenlin Zhang, Yimin Deng, Shirong Wang, Jinyang Li, Xian Wu, Xiangyu Zhao
cs.AI
2026-11-09
SoMem keeps agent memory as editable Markdown linked to raw dialogue. On LoCoMo, BEAM-100K, and LongMemEval-S it beats the best baseline by 2.66, 3.10, and 2.00 points.
After deployment the weights stay fixed, so preferences, task progress, and corrected facts have to live outside the model. Most memory systems lock the record format first: short extracted facts, or a fixed consolidation hierarchy. That format is awkward once a later turn rewrites an old fact or joins two episodes.
Keeping the full history preserves detail and turns the next question into a search. Compressing it speeds reading and drops qualifications. SoMem, short for self-organized memory, splits the job. The model decides how to group and revise text. The runtime keeps the evidence and the indexes consistent.
Memory is a folder of Markdown files. Each new session is appended to Source and then frozen. The Writer, the model that updates memory, may cite Source and may not edit it. It proposes changes to Topic documents, files grouped by person or task: create, rewrite, merge, move, or promote a standing preference into Core. Core is a size-capped slice of Topic, inserted at the start of every question, in the slot a system prompt usually fills.
Edits are staged. The runtime accepts them only if Source is untouched, source references resolve, links point at paragraph ids, Core stays within capacity, times are valid, and paths remain inside the workspace. A rejection leaves the previous state in place. An acceptance is installed atomically, and three views are rebuilt. Graph turns paragraph links into outgoing edges and backlinks, with no separate relation extractor. Timeline files group statements by event time at year, month, or day precision. Recent holds the latest created units and is preloaded beside Core.
A paragraph is the memory unit. A footnote stores the event time and the Source ids. A stable id lets other files link to that paragraph.
At question time the Reader, the model that answers, starts with Core, Recent, and a file inventory, plus the question date when one exists. The only tools are search, list, and read. Each search is BM25 or dense similarity, with optional path and date filters, and the model picks the mode for that query. The model also decides what to open next. If a summary is thin, the Reader follows links into Topic and opens Source to check the original lines. The same model can write and read. A different model can reuse the workspace as it stands.
In the main comparison GPT-6 Luna is both Writer and Reader, with no extra reasoning effort. GPT-4o-mini at temperature 0 is the judge. The field is A-Mem, Mem0, MemoryOS, Nemori, LightMem, E-mem, and Full Context, which reads the entire untruncated history with no retrieval.
| Benchmark | Metric | SoMem | Best comparison | Gap |
| LoCoMo, 1,540 questions | LJ accuracy | 88.96 | E-mem 86.30 | +2.66 |
| BEAM-100K, 400 questions | nugget score | 54.02 | Full Context 50.92 | +3.10 |
| BEAM-100K | pass rate | 61.50 | A-Mem 55.75 | +5.75 |
| LongMemEval-S, 500 questions | LJ accuracy | 76.50 | LightMem 74.50 | +2.00 |
On LoCoMo, single-hop is 94.53 against 94.17 for Full Context, and temporal reasoning is 88.16 against 82.24 for Mem0. Open-domain is 59.38, a slim edge over E-mem at 58.33, while Full Context falls to 45.83. Multi-hop is 83.33, behind E-mem at 85.11.
On BEAM-100K the gains sit in temporal reasoning (53.75 versus 41.25 for Full Context), event ordering (46.84), and instruction following (73.75, tied with Nemori). Contradiction resolution is 36.56 against 51.56 for E-mem. Abstention is 23.75 against 42.50 for MemoryOS. Information extraction is 73.70 against 82.86 for the full history. The average is higher. Several capabilities are not.
On LongMemEval-S, assistant single-session is 99.11 and user single-session is 98.57, with temporal reasoning at 73.31, all best in the table. Knowledge update is 77.56, against 91.67 for LightMem and 92.31 for Full Context. Multi-session reasoning is 58.65, against 75.56 for LightMem. Keeping both sides of the dialogue helps when the question needs the original wording. It helps less when the question is which earlier fact has been retired.
The LoCoMo ablation freezes memory and only changes what the Reader can see, on all 1,540 questions. Dropping Source cuts overall accuracy from 88.96 to 80.32. Dropping Topic cuts it to 86.04. Paired confidence intervals for both exclude zero. Dropping Core leaves the score at 88.96. This set barely tests standing preferences. Dropping Recent moves single-hop from 94.53 to 92.51 and temporal accuracy from 88.16 up to 89.10. Dropping Timeline or Graph each brings temporal accuracy to 86.92. Core and any single derived view stay inside sampling noise on the overall number.
A finished workspace transfers. Memory written by GPT-5.6 Luna and read by Qwen 3.8 Flash scores 93.25 on LoCoMo, above the 92.34 from giving that same memory back to GPT-5.6 Luna.
More reasoning effort usually lifts scores, with steadier gains on BEAM's longer histories. The curve saturates and sometimes reverses. On BEAM-100K, moving GPT-6 Luna from High to xHigh raises quality while cutting searches, file reads, calls, and cumulative tokens.
Once writing is separated from reading, the workspace is the artifact. Topic files are ordinary Markdown. Graph, Timeline, and Recent are derived after install, so a new Reader does not rebuild memory. With the shared reader, the organized store also beats stuffing the raw history into context: 88.96 versus 84.22 on LoCoMo, and 76.50 versus 65.80 on LongMemEval-S.
The margin is incremental. With one GPT-6 Luna and no extra reasoning, the three overall leads are 2.66, 3.10, and 2.00 points, and several subtasks go the other way. What remains useful is the division of labor: the model edits the prose, and code guards consistency. Another model can open the same document store.
The benchmarks are question answering over conversation logs. They do not cover long-running personalization or actual task execution. Reasoning-effort sweeps use fixed subsets, 100 LoCoMo questions and 200 BEAM questions, and each condition is a single run. The judge is an LLM with no systematic human calibration. Ablations close an access path on a frozen memory. They do not ask the model to reorganize after a component is removed. The runtime checks references and structure. It does not check whether the prose is right or complete. Catching omissions on later updates is left open. A deployment still needs access control, retention rules, and isolated tool execution.
Rows with other backbones are kept out of the ranking. With DeepSeek V4 Flash as both Writer and Reader, SoMem reaches 90.20 on LongMemEval-S and 92.73 on LoCoMo. The other methods were not rerun on that backbone, so those scores are not a comparison with LightMem at 74.50. Abstention and contradiction resolution on BEAM are weak, and the paper does not separate out why. Construction and maintenance cost is not lined up against the baselines in one table.