Tencent releases SAPS: sparse attention that learns KV block selection end-to-end
teortaxesTex · x · 2026-09-15
Tencent has released a new sparse attention method, Simple Attention Sparsification (SAPS), on Hugging Face.
The key idea: instead of relying on handcrafted heuristics for which KV blocks to attend to, SAPS lets the language modeling loss directly optimize block selection per query — making sparse attention end-to-end learnable and potentially cutting long-context inference cost.
Related event: Tencent Open-Sources SAS Sparse Attention Method(4 posts)→
More from Research
- ELLIS x UniReps talk: why distinct learning processes yield similar representations — ClementineDomi6 · 2026-10-30
- Physicist calls OpenAI's Navier-Stokes solution physically meaningless despite loophole win — GaryMarcus · 2026-10-30
- Anthropic paper: 0.1% pretraining poison plants RCE backdoor in 4B CLI agent — omarsar0 · 2026-10-30
- CopeNLU presents 7 papers at EMNLP 2026 on bias, alignment, explainability — IAugenstein · 2026-10-30
- Two-stage 3D ResNet pipeline detects neonatal brain pathologies from MRI with 0.919 AUC — Dr_Alex_Crimi · 2026-10-30
- 40 years of putting new neurons into the CNS: why so slow, and can AI accelerate it? — patrickmineault · 2026-10-30