Continual learning log: SGD + PCA replay hits 91.1%, full code released
BlackHC · x · 2026-09-16
The author wraps up a systematic experiment ladder on a tiny continual learning benchmark and releases the full research log with plots and code: SGD + PCA replay at 4 epochs per task reaches 91.1%, versus 95.2% for joint training with 10 passes. Key takeaways: check what your evaluation hides, treat loss scaling as part of the optimizer setup, and Adam vs SGD held surprises.
Related event: BlackHC's toy continual learning experiment: PCA replay hits 91.1%(6 posts)→
More from Research
- Well-funded AI4Science startups bet epistemology is optional for doing science — ludwigABAP · 2026-09-16
- An interactive Margolus-neighborhood cellular automaton where sand falls and water flows — mhmazur · 2026-09-16
- antirez open-sources DwarfStar's fused inference kernels under MIT — antirez · 2026-09-16
- The unreasonable effectiveness of BM25 for agentic search — Jo Kristian Bergum, Hornet.dev — AI Engineer · 2026-09-16
- Berkeley team shows probes can steer models to any level of risk aversion — soumitrashukla9 · 2026-09-16
- Recurrent Looped Transformer feeds final hidden states back for deeper reasoning — WebAssemblyMan · 2026-09-16