Xiaohongshu's DoSA cuts 1M-context TTFT by 37-48% by speeding up sparse-attention indexer

机器之心 · wechat · 2026-10-20

Xiaohongshu's Dots team has built DoSA (Dots Sparse Attention), which accelerates the indexer that selects tokens in sparse attention while preserving the original Top-K selection, filtering about 97% of candidates in tests.

Problem: DeepSeek-V3.2's DSA has an indexer score tokens before main attention, but the indexer itself becomes the bottleneck. At 1M token context, indexer time accounts for 81.6%, 73.0% and 90.8% of prefill time for Dots3-Note-preview, GLM-5.2 and DeepSeek-V3.2 respectively.

Method: DoSA keeps the original scoring function and uses mathematical bounds to prove which computation can be skipped. Seed candidates are rescored with all index heads to get a lower bound on the Top-K threshold; each candidate then gets an upper bound on its true score (a few important heads computed exactly, positive and negative weight heads bounded conservatively). Only candidates whose upper bound is strictly below the threshold are pruned; the rest are scored exactly. The pipeline is organized into regularized filtering, exact scoring and compact write-back, exploiting candidate overlap across queries.

Results: On GLM-5.2's full indexer microbenchmark, DoSA beats SGLang and LiteTopK in all 60 configurations, with 2.47-2.74x speedup over SGLang at 1M context. End-to-end TTFT at 1M context drops about 37%, 46% and 48% versus SGLang baselines for Dots3-Note-preview, GLM-5.2 and DeepSeek-V3.2.

Original post →

Xiaohongshu's DoSA cuts 1M-context TTFT by 37-48% by speeding up sparse-attention indexer

More from Infra