Meta's Self-Improving Memory: agents discover their own memory skills and distill them into weights
jaseweston · x · 2026-11-17
Meta AI's RAM team (Jason Weston, Weizhe Yuan, et al.) introduces Self-Improving Memory, letting a model discover its own memory skills and train them into its weights for continual-learning agents.
Two stages
- Harness Optimization (discover): a meta agent reads task-agent trajectories, finds where memory use (LOAD/OFFLOAD prompts) breaks down, and rewrites the prompts, keeping the best-scoring candidates. Across three Continual Learning Bench tasks, optimized prompts average 56.0 vs 44.6 for the best hand-written ones and 36.1 generic (+11.4 avg; +20.6 on Codebase).
- Harness Distillation (demonstrate + distill): run the agent with optimized prompts, swap them back to generic in the recorded runs, then LoRA fine-tune. With only generic prompts at test time the model scores 43.1 vs 37.6 for SFT on generic runs.
Notably, the meta agent and task agent are the same model (Kimi-K2.6) — the model learns memory skills it found itself.
Related event: Meta's Self-Improving Memory Lets Agents Discover and Distill Memory Skills(4 posts)→
More from coding & agent
- How Comet improved a financial incident remediation agent's score from 0.34 to 0.57 with Opik — dl_weekly · 2026-11-17
- Jason Weston: Harness Optimization Hits 56.0, Distillation Adds 5.5 More — jaseweston · 2026-11-17
- Every shows Claude Fable 5.2 building interactive games — slow but token-hungry — every · 2026-11-17
- Two free Wan2.1 ComfyUI workflow sets: native nodes vs Kijai wrapper with speed boosts — asdfghmfker · 2026-11-17
- Keeping filler words in prompts encodes real uncertainty and nails it in one shot — brandon_galang · 2026-11-17
- Will Brown essay: strongest model may be the wrong teacher — SFT, RL and on-policy distillation — heghbalz · 2026-11-17