SPARE preprint: 2.55x faster decoding for linear attention on a 1.3B model

krishnan · x · 2026-11-17

krishnan breaks down SPARE, a Nov 5 preprint from University of Virginia targeting the second cost of linear attention: models still read/rewrite a large recurrent state each decoding step. SPARE keeps each head's state in a fixed smaller subspace, uses offline calibration for projections with per-head capacity, adds light write compensation, and requires no retraining. On a 1.3B Gated DeltaNet at batch size 512 on RTX A6000 GPUs, the most compressed setting decoded 2.55x faster. The authors note the projection is not lossless — test speed and recall separately.

Original post →

More from Infra

Infra channel →