STABLE Optimizer Accepted at NeurIPS 2026: Taming Continual Learning Forgetting at the Optimizer Level
v_lomonaco · x · 2026-11-11
A new paper, STABLE: A Continual Learning Optimizer with Adaptive Drift Control, has been accepted at NeurIPS 2026 by researchers from the University of Pisa, CNR-ISTI, and LUISS, with code released.
- Key insight: catastrophic forgetting stems from unstable learning dynamics and can be addressed at the optimizer level, rather than via replay buffers, architecture changes, or extra regularization.
- Theory: they formally prove that for distance-based classifiers, bounded parameter drift guarantees bounded margin erosion under a local score-stability condition.
- Method: STABLE is a primal-dual optimizer enforcing a drift budget—raising the dual variable when the budget is violated and lowering it otherwise—yielding high stability without penalizing plasticity.
- Results: on long-horizon unimodal streams and CLIP-based multimodal streams, vanilla PEFT baselines trained with STABLE match or beat specialized continual learning methods while preserving zero-shot transfer.
More from Research
- ARLoop releases fully open looped transformer recipe; 1.2B model beats Huginn-3.5B — Lianhuiq · 2026-11-11
- UniPath (NeurIPS 2026): adaptively coordinating understanding and generation in one model — jindong_wang92 · 2026-11-11
- Scale AI's 1,200-run study: PPO wins at small batches, GRPO catches up at scale — heghbalz · 2026-11-11
- Open-source effort designs personalized pancreatic cancer vaccine for 34-year-old patient — jajoosam · 2026-11-11
- Ross Girshick posts CVPR slides recounting 1.5 decades of object detection and its link to the LLM era — CatAstro_Piyush · 2026-11-11
- KAIST's WikiTrace: evidence-linked memory fixes agent "memory-induced degradation" — dair_ai · 2026-11-11