Tencent Open-Sources SAS Sparse Attention Method

Tencent's Hunyuan team open-sourced SAS (Simple Attention Sparsification) on Hugging Face, using continuous gating inside attention softmax to end-to-end optimize context ranking with the language modeling loss, extending context length on Qwen3-based models.

2026-09-14 ~ 2026-09-15 · 4 related posts

• Tencent Hunyuan Open-Sources SAS: End-to-End Sparse Attention via Context Ranking — Tencent-Hunyuan · 2026-09-14
• Tencent releases Simple-Attention-Sparsification: sparse attention for long-context Qwen3-14B — tencent · 2026-09-14
• Tencent Hunyuan's SAS optimizes sparse attention end-to-end via gating inside attention softmax — _akhaliq · 2026-09-14
• Tencent releases SAPS: sparse attention that learns KV block selection end-to-end — teortaxesTex · 2026-09-15