Meta's Self-Improving Memory: discovered prompts beat hand-written 56.0 vs 44.6, distillation adds 5.5 points
jaseweston · x · 2026-11-17
Closing posts of Jason Weston's Self-Improving Memory thread, with full blog details:
- Harness optimization: on three Continual Learning Bench tasks, prompts the model discovers score 56.0 vs 44.6 for the best hand-written ones (+11.4);
- Harness distillation: run with optimized prompts, swap back to generic while keeping responses, then LoRA fine-tune. At test time with only generic prompts: 43.1 vs 37.6 for SFT on generic runs (+5.5);
- Case study: the meta agent noticed one ever-growing summary file, so it added instructions to save repo paths, append a rule after each failure, and keep a symptom-to-fix index — lifting the Codebase task from 33.5 to 55.7 in two iterations.
Blog: facebookresearch.github.io (RAM @ Meta AI).
Related event: Jason Weston Boosts Agent Memory via Harness Optimization and Distillation(2 posts)→
More from coding & agent
- How Comet improved a financial incident remediation agent's score from 0.34 to 0.57 with Opik — dl_weekly · 2026-11-17
- Jason Weston: Harness Optimization Hits 56.0, Distillation Adds 5.5 More — jaseweston · 2026-11-17
- Meta's Self-Improving Memory: agents discover their own memory skills and distill them into weights — jaseweston · 2026-11-17
- Every shows Claude Fable 5.2 building interactive games — slow but token-hungry — every · 2026-11-17
- Two free Wan2.1 ComfyUI workflow sets: native nodes vs Kijai wrapper with speed boosts — asdfghmfker · 2026-11-17
- Keeping filler words in prompts encodes real uncertainty and nails it in one shot — brandon_galang · 2026-11-17