New Paper Explains Mamba-3's State Rotations as Underdamped Second-Order Dynamics
tri_dao · x · 2026-10-13
- A new arXiv paper, Understanding Mamba-3 Through Oscillatory Dynamics, interprets Mamba-3's input-dependent recurrent state rotations as underdamped second-order dynamics: decay controls the damping envelope while phase shapes oscillation.
- Controlled comparisons show underdamped (oscillatory) dynamics beat non-oscillatory damping regimes on state-tracking tasks; LM pretraining shows benefits for long-context prediction and length generalization.
- Interventions in pretrained models confirm learned phase matters; state analysis suggests rotation regulates how retained information combines with incoming updates.
- Conclusion: oscillation is a useful mechanism for recurrent computation, supporting long-context processing by governing information accumulation. The accompanying figure contrasts Mamba-2's first-order exponential decay with Mamba-3's oscillatory dynamics.
More from Models
- autotrust's JEV-35B-NVFP4, a Qwen3.5 MoE-based decision model, hits HF trending — autotrust · 2026-10-13
- Theo benchmarks Opus 5.5, GPT-6.1 Sol and Muse Wonder Full on a bug fix — shadcn · 2026-10-13
- Claude Code at $100/month plus $20 ChatGPT: AI subscription costs stack up — dumpshoot · 2026-10-13
- Grok subscription is currently offering a 50% discount — sujingshen · 2026-10-13
- Pleias team on leaderboards: build internal evals for real use cases, but can't ignore them — antoine_chaffin · 2026-10-13
- Redditor proposes refillable rolling compute tank to replace OpenAI's 5-hour usage resets — PastaPandaSimon · 2026-10-13