Xiaohongshu's DoSA cuts 1M-token TTFT by up to 48% with lossless sparse-attention indexer speedup
小红书技术REDtech · wechat · 2026-10-21
Xiaohongshu's Dots team released DoSA, tackling the bottleneck where the indexer in sparse attention (e.g., DeepSeek-V3.2's DSA) dominates long-context prefill—at 1M tokens it consumed 73–91% of prefill GPU time across three tested models.
Key idea: unlike approximations (layer reuse, fewer index heads) that can change Top-K selections, DoSA provably skips candidates that cannot make Top-K. It builds a threshold lower bound from the K-th best score among seed candidates, then constructs mathematically guaranteed upper bounds per candidate—only strictly-below-threshold candidates are excluded, keeping selection lossless.
Results: 97% of query-key candidate pairs skip full-head scoring on GLM-5.2; indexer speedups of 1.12–2.74x vs SGLang; end-to-end TTFT at 1M tokens reduced 37% (Dots3-Note-preview), 46% (GLM-5.2) and 48% (DeepSeek-V3.2). Open-sourced as dosa-sglang; paper arXiv:2610.13990.
Related event: Xiaohongshu Open-Sources DoSA, Cutting First-Token Latency by Up to 48%(2 posts)→
More from Infra
- July 28 MCP release makes remote servers stateless, biggest revision since launch — bibryam · 2026-10-23
- MCP keynote: AAIF hits 305 members, Skills over MCP and Tasks/Events roadmap revealed — bibryam · 2026-10-23
- Only 8% of cloud vendors support OIDC — dev calls to kill long-lived API keys — cramforce · 2026-10-23
- Swift-1.5-Qwen3.8-27B ported to hipfire MQ4: 2.2x faster end-to-end than GGUF on a 7900 XT — Queasy_Asparagus69 · 2026-10-23
- Intel CTO: chip output can at best 2X while AI compute grows 3X per year — SammaVaco · 2026-10-23
- Engineer reverse-engineers projector TV's 10s wake lag to a cold-booting Linux board — banteg · 2026-10-23