New paper says scaling long-term memory beats scaling the backbone, with 39% fewer parameters
burny_tech · x · 2026-08-04
Memory modules scale better than backbone size in a new long-term memory paper
The paper "Memory Decoder at Scale: A Pretrained, Parametric Long-Term Memory" argues that long-term memory can be scaled separately from the base decoder. The authors train a parametric memory to imitate kNN retriever distributions offline, then combine it with a frozen decoder via next-token probability interpolation.
Key results:
- A Pythia-410M base model plus a 6.9B memory beats Pythia-12B on 17 tasks while using 39% fewer total parameters.
- The approach also works across domains: 1.7B domain memories improve Qwen3 Base models by more than 9 points on average in biology, law, and finance.
- Because indexing and retrieval at this scale are expensive, the authors also introduce a distributed Faiss pipeline and sparse batch-wise loading of kNN distributions.
Overall, the work suggests that independently scaling pretrained memory may be a more parameter-efficient path than scaling the backbone alone.
More from Research
- A model-stealing paper’s proof needs a stronger spanning assumption, the author says — ArthurConmy · 2026-08-04
- Model-stealing paper author says the original theorem was wrong, but the result can be repaired — ArthurConmy · 2026-08-04
- RedMonk says only 1% of surveyed open source code is machine-written — rseroter · 2026-08-04
- Robot walking demo adds target-facing rewards but still fails 80% of the time — carlosdponx · 2026-08-04
- Airbnb details its internal AI eval stack, sampling 5% of traffic daily — econoar · 2026-08-04
- Aero Hand Open debuts as a $314 open-source robotic hand with 16 joints — TinfoilTricorn · 2026-08-04