CRUG lets a single RNN learn new dynamical systems with zero forgetting by recycling units
tweetsatpreet · x · 2026-10-02
A new arXiv paper introduces CRUG (Continually-Recyclable Unit-Gating) for continual dynamical systems reconstruction in recurrent neural networks.
Key ideas:
- Benchmarks parameter regularization, replay, and parameter isolation on the interpretable Almost-Linear RNN (AL-RNN)
- Parameter isolation preserves old dynamics best but quickly exhausts a fixed-size network via task-specific allocations
- CRUG uses differentiable gates trained with an L0-based penalty to select task-specific units and recycles unused units for later tasks; directed connections let new tasks reuse earlier representations without disturbing committed dynamics
Results: Best reconstruction-capacity trade-off among tested methods, zero forgetting, reliable learning of heterogeneous nonlinear and chaotic systems, plus notable forward transfer.
More from Research
- DeepSeek's DSec paper: 3M sandboxes/day, 380K concurrent, powering V3.2-V4.1 RL — jiqizhixin · 2026-10-02
- Build an eval: the real path to understanding model performance and training data — divy93t · 2026-10-02
- Eric Topol: evidence from 105,000-woman MASAI trial says all mammograms should include three AI algorithms — EricTopol · 2026-10-02
- Extropic uses RL post-training to accelerate thermodynamic algorithm discovery, first sparks of Thermo RSI — beffjezos · 2026-10-02
- Optimized SuperPoint hits 2.1x faster keypoint detection on Jetson Orin Nano — sdiazlor · 2026-10-02
- Mathematician Kevin Buzzard: math community is 'grieving' through five stages over AI — littmath · 2026-10-02