Regularity and Stability Properties of Selective SSMs with Discontinuous Gating
Nikola Zubić, Davide Scaramuzza
cs.LG, math.DS, math.OC, stat.ML
2025-05-17
Mamba's online gating leaves stability murky. Passivity and ISS analysis yields a differentiable regularizer that cuts Mamba-core instability about 92% across 7 time-series datasets at near-zero accuracy cost.
Selective state-space models (selective SSMs), Mamba being the headline example, are now central to long-sequence modeling. Their stability, however, is poorly understood. Their state-space coefficients get modulated online by a token-dependent gating signal, so the recurrence is neither linear time-invariant nor classically nonlinear, and prior analyses, built on expressivity or Lyapunov-exponent arguments, give no unified energy-based account. This paper supplies the missing stability theory and turns it into a usable training regularizer.
The authors analyze continuous-time selective SSMs through passivity, dissipativity, and Input-to-State Stability (ISS), the classical tools of control theory. The decisive move is separating two input classes: the selection signal x(·) schedules the system dynamics through the A, B, C matrices, while the driving input u(·) enters the supply rate. This separation is what lets the analysis capture "gating changes online."
Four main results. First, under strict dissipativity homogeneous trajectories forget exponentially (Theorem 4.1). Second, once the selection is frozen (x≡0), the subsystem is a genuinely passive linear time-varying system whose minimal available storage is necessarily quadratic, with AUCloc regularity holding even under discontinuous gating (Theorem 4.2), a condition needed to handle piecewise-constant gates. Third, under a universal quadratic storage hypothesis, the paper gives a parametric LMI (linear matrix inequality), universal kernel constraints, and an "irreversible forgetting" property: once a state direction becomes energy-less, no selection can ever make it observable again. Fourth, it gives sufficient conditions for global ISS uniformly over all admissible selection schedules (Theorem 7.1).
The bridge to practice comes in Section 6: a one-step discrete dissipativity inequality and a sampled block LMI tailored to Mamba's selective-scan recurrence. The largest positive eigenvalue of this matrix is differentiable with respect to model parameters, so it drops in directly as a training-time regularizer.
Experiments run on 7 standard time-series datasets (ETTh1/h2, ETTm1/h2, weather, electricity, traffic) across 4 prediction horizons each, 28 dataset-horizon pairs in total.
With the LMI regularizer added, mean Mamba-core LMI violations drop about 92%, and all 28 pairs improve. The cost is tiny: clean MSE stays within 0.018% of the unregularized baseline. In a conservativeness ablation, making the diagonal matrix P=diag(q) learnable Pareto-improves over P=I on all 28 pairs at no measurable accuracy cost.
One "neutral result" deserves mention. In the routing ablation (Section 8.5), the penalty consistently pushes the Mamba branch weight down, but the train-time correlation between the Mamba weight and local LMI violation is essentially zero in aggregate and inconsistent in sign across datasets. The regularizer does suppress Mamba-core instability; it does not exhibit an automatic effect of routing traffic away from the unstable Mamba branch.
For anyone training long-sequence models on Mamba-style architectures, this paper offers a theoretically backed regularizer: it cuts Mamba-core numerical instability sharply (about 92%) at almost no accuracy cost (0.018%), and it is differentiable and plug-and-play. More important, it turns "is Mamba stable?" from an empirical question into a computable, monitorable quantity, the sampled LMI violation, which you can watch directly during training.
The authors scope their claims carefully, which is the honest part and also the limitation. First, the criterion targets the Mamba selective-scan core, not a passivity certificate for the full network with normalization, residuals, and routing. Second, it uses a sampled, locally linearized criterion, not exact discrete passivity. Third, it is validated on a single architecture family (SST/Mamba); generalization to other selective SSM variants is unestablished. Add the "neutral result": the regularizer suppresses core instability but does not automatically reroute traffic away from the unstable branch, which signals a gap between this theory and whole-network training dynamics.