SPARE preprint: 2.55x faster decoding for linear attention on a 1.3B model
krishnan · x · 2026-11-17
krishnan breaks down SPARE, a Nov 5 preprint from University of Virginia targeting the second cost of linear attention: models still read/rewrite a large recurrent state each decoding step. SPARE keeps each head's state in a fixed smaller subspace, uses offline calibration for projections with per-head capacity, adds light write compensation, and requires no retraining. On a 1.3B Gated DeltaNet at batch size 512 on RTX A6000 GPUs, the most compressed setting decoded 2.55x faster. The authors note the projection is not lossless — test speed and recall separately.
More from Infra
- Q-CTRL unifies calibration and execution into autonomous workflow, QPUs ready in hours — MJBiercuk · 2026-11-17
- Celesto, a CI built for coding agents, claims 90% cheaper and 25% faster builds — aniketmaurya · 2026-11-17
- Reddit debate: US chip bans accidentally saved China from the AI capex bubble — PureControl3423 · 2026-11-17
- Quail adds hybrid GDN model support, beating vLLM by 3.2x on long-doc queries — sh_reya · 2026-11-17
- SGLang Summit: Hyperbolic argues idle GPUs are the other half of inference optimization — BanghuaZ · 2026-11-17
- Researchers Find Undocumented x86 Instructions Allowing User-Space Microcode Writes — sull · 2026-11-17