Open-source Latent Sparse Attention hits full-attention accuracy, up to 22.7x faster
TheTuringPost · x · 2026-11-07
- The team open-sources Latent Sparse Attention (LSA), a step toward subquadratic attention that matches full-attention accuracy at ultra-long contexts.
- Key insight: sparse attention attends to only a few tokens per query, but at 10M tokens the search for the right ones takes 98% of the attention layer's time.
- LSA tackles this retrieval bottleneck, speeding up sparse attention by up to 22.7x, framed as a promising path to scaling AI efficiently at extreme context lengths.
More from Infra
- BUI decouples Blender's GPU UI kernel into a standalone runtime: 18MB memory, 12ms cold start — Obijuan_cube · 2026-11-07
- TSMC unveils BEOL oxide-semiconductor memory: 0.75V operation, sub-8ns R/W cycles — Terminator857 · 2026-11-07
- PJM data: data-center demand drove a minority of 2026 power cost hikes; electricity itself rose 47.7% — kimmonismus · 2026-11-07
- Qualcomm's NeurIPS paper: Leech lattice vector quantization compresses LLMs without codebooks — HowDevelop · 2026-11-07
- Box CEO Aaron Levie: long-running AI agents let inference run anywhere, reshaping data center economics — inductionheads · 2026-11-07
- Cloudflare details Vole-Mole, its path to web privacy in a post-quantum future — AIWithLucas · 2026-11-07