Compact-Memory LLM Agents via Online Max-Member Clustering and Atom-Aware Packing
Jiahe Geng, Jinpeng Wang, Kun Yuan
cs.AI
2026-09-04
RSM-full is online clustered memory for compact LLM agents. At a 4k AMA-Bench budget it keeps 83% of full-context quality at 32% of the tokens, +3.5-6.0 vs K-Means.
Long-horizon agents cannot keep the full transcript in the prompt. Latency, cost, and windows all blow up. Fixed windows and top-k retrieval cap length and often drop the latent structure that makes old experience reusable. The live question is not raw recall. It is which memory design wins the quality–token trade in the roughly 2k–5k prompt band.
Unlike ACON or MemGPT, which decide what to compress or summarize, this paper stays at the vector-index and prompt-assembly layer: how streaming memories merge, and how retrieved snippets are packed.
RSM-full has two load-bearing rules. Higher-rank bases and quantization live in the appendix.
Writes use a cosine-gated max-member merge. A new chunk is assigned to the atom (cluster) with maximum cosine. If that cosine against the atom mean or any member clears a threshold τ, it merges; otherwise a new atom is allocated. The extra member check is more permissive than centroid-only gating, so similar chunks are less likely to split because the mean drifted. τ is set per embedding space as about the 70th percentile of pairwise cosines on ≤50 unlabeled validation items; on AMA/BGE it is 0.85.
Reads use an atom-aware grouped packer. Top-K atoms are retrieved, members ranked inside each atom, then written into the prompt grouped by atom with headers and temporal order, not as a flat relevance list. Retrieval may use the top singular vector or the normalized centroid; on BGE-normalized AMA, once packing is fixed, the two do not differ reliably.
AMA production settings: chunkturns=5, retrievek=6, k=8, about 4k tokens, agent GPT-4o-mini.
AMA-Bench has 208 episodes and 2,496 QAs. At 4k tokens RSM-full scores 0.311 against Full-Context 0.373 at 12,519 tokens: 83% of the quality at 32% of the tokens. Budget-RAG is 0.292, Streaming-Proto 0.302, Online K-Means 0.281, reproduced A-MEM 0.273. Across four seeds and three budgets (2.6k / 4k / 5k) the gap to K-Means is +3.65 / +3.50 / +6.04, all p<.001.
| Method | Score | Tokens | vs Full-Context |
| Full-Context | 0.373 | 12,519 | 100% |
| RSM-full | 0.311 | 4,001 | 83% quality / 32% tokens |
| Streaming-Proto | 0.302 | 3,982 | n/a |
| Budget-RAG | 0.292 | 4,146 | n/a |
| Online K-Means | 0.281 | 4,000 | n/a |
| A-MEM | 0.273 | 4.2k | n/a |
A 2×3 factorial splits the gain. With packing fixed, max-member merge is +5.74 vs K-Means and +5.45 vs matched-τ DP-means. The retrieval-rule contrast is p=0.40. With clustering and retrieval fixed, grouped packing is +5.02±1.00 over flat concatenation. Giving K-Means the same packer still leaves RSM +4.90.
RealMem (10 personas, 1,415 unique QAs) repeats the pattern: RSM-full 0.4684, +0.69 vs Budget-RAG (p=0.006), +1.65 vs A-MEM, +2.97 vs Streaming-Proto. The +0.27 vs BM25-RAG is not significant; the authors do not claim equivalence. The effect is smaller than on AMA, consistent with weaker atom structure and stronger lexical cues.
Under tight budgets, the first-order memory knobs are the merge rule and the packer, not a fancier LLM organizer. Each is a small change at the index layer, and each is worth about five points. Around 4k tokens, when the stream has repeated structure, this is a clear Pareto point. Above about 12k tokens, full context or lexical retrieval still wins.
Int8 matches float32 on AMA (0.315 vs 0.311). Appendix storage machinery is a cross-geometry path, not the reason for the headline result.
Positive end-to-end claims are scoped to AMA-Bench and RealMem. Raw LoCoMo and LongMemEval are boundaries: dense retrieval loses to BM25, RSM-full collapses toward centroid-equivalent behavior, and the compact-memory edge fades. The v1 retrieval rule fires on 2 of 4 LoCoMo-Plus relation families and is not a general solver.
There is no selective forgetting. One embedder and one primary judge stack (BGE + GPT-5.4 rejudge); cross-embedder replication is untested. On AMA-Long at N=32k every method sits on the no-memory ceiling, so the long-horizon ablation is about atom count, not answer quality. MemGPT and MemoryBank fall to or below no-memory on RealMem; the authors flag reproduction issues and should not be used as method-level ceilings.