Composing continual learning mechanisms lifts retention from 1.2% to 34.9%
DanielKhashabi · x · 2026-09-09
Johns Hopkins researchers release ComposeCL, showing that simply composing existing continual learning mechanisms dramatically extends long-horizon memorization.
- Combines weight anchoring (Synaptic Intelligence), function anchoring (self-distillation), data anchoring (generative replay), and merged LoRA in a full 2⁴ factorial experiment across 100 sequential tasks per dataset, 3 seeds
- Best composition raises average final retention from 1.2% (naive sequential SFT) to 34.9% — a 28-fold improvement — ranking top-3 on all three datasets
- Replay and merged LoRA give the largest gains, with joint benefit exceeding the sum of individual gains on all datasets
- Paper, code, and datasets are open-sourced (arXiv, GitHub, alphaxiv)
More from Research
- New EMNLP paper uses controlled rearing to explain how learners reject ungrammatical "X laughed Y" — EliasEskin · 2026-09-10
- Greg Kamradt digs into gamedevbench: reference solutions edit 4.7 files, 114 lines on average — GregKamradt · 2026-09-10
- Ex-Continue founder publishes 'The Superdark Factory' in MIT Press: 35T agent tokens/month by 2029 — tylerjdunn · 2026-09-10
- Experiment with 100 AI agents: when 9% cheated on math problems, 24% blew the whistle — weballergy · 2026-09-10
- Scientific Imaging and ML workshop set for San Juan, March 2027 — applications open until Oct 2026 — prof_kamilov · 2026-09-10
- Lesioning one neuron in a fruit fly brain model makes it misidentify emotions — emax · 2026-09-10