JSSA sparse attention hits 6.43x decoding throughput at 128K context on B200 GPUs
SNU-VLSI · hf · 2026-10-15
Seoul National University researchers introduce Joint Subspace Sparse Attention (JSSA), tackling the KV-cache traffic bottleneck in long-context LLM decoding.
- Token-level sparse attention faces two challenges: low-dimensional scoring can lose query–key structure needed for accurate ranking, and gathering scattered tokens can offset latency gains
- JSSA learns a shared low-dimensional query–key subspace via a tractable marginal objective with a closed-form spectral solution
- A gather-free token-level indexed-attention kernel supports distinct per-KV-head selections under standard GQA KV-cache layouts
- Across GQA, hybrid sliding-window/global, and MLA models, JSSA stays near full-KV accuracy; on an NVIDIA B200 it achieves up to 6.43x decoding throughput at 128K context
- Without training it matches model-native DSA reasoning accuracy and reaches 96–99% of its decoding throughput even at 700B scale
More from Infra
- patio11 on multi-agent dev: clearly the future, and we'll buy so much inference — IgorCarron · 2026-10-15
- Hugging Face Hub makes model tensor inspection up to 20x faster, DeepSeek-V3.2 drops from 9.2s to 0.45s — victormustar · 2026-10-15
- RTX 5060 Ti 16GB surges up to 144% in Japan, hitting $1,292 as RTX 5080 tops $2,400 — chemist_slime · 2026-10-15
- Fish Audio Drama 3 lands in vLLM-Omni: 128 concurrent streams on one H200 — vllm_project · 2026-10-15
- 1,174 Macs form Darkbloom inference network, growing 4.5x in 60 days at half the cost — gajesh · 2026-10-15
- ByteDance in advanced talks with MediaTek for custom AI data-center chips, Bloomberg reports — mark_k · 2026-10-15