SketchSSM cuts SSM state traffic 11x at sketch rank 8 with near-lossless accuracy

sehoonkim418 · x · 2026-10-08

Headline result: At mean sketch rank 8, SketchSSM reduces state traffic in SSM-style models by 11x while keeping near-lossless accuracy on multiple reasoning and recall benchmarks. The author notes pruning and quantization fall apart well before that point.
Generality: The method works across Mamba-2, Gated DeltaNet, and KDA architectures.
How it works: Within a window the state is fixed and only queries change, so queries are compressed onto a small low-rank basis and the state's outputs are computed once for that basis. Every query is a mix of the basis, so its output is just the same mix of the stored outputs—no full-state read required.

More from Infra