Sol-Attn: Accelerating Video Generation Inference via On-the-Fly Attention Sparsification
Haopeng Li, Yitong Li, Junsong Chen, Tian Ye, Haozhe Liu, Jincheng Yu, Duomin Wang, Ruihua Zhang, Zeke Xie, Enze Xie, Song Han
cs.CV
2026-07-27
Sol-Attn folds sparse-attention routing and approximate correction into one online-softmax pass, speeding video generation 2.1-3x (5.08x inside Sol-Engine) with no quality loss.
Diffusion transformers drive high-fidelity video generation, but their token sequences run into the tens of thousands, and attention scales quadratically with length. That makes attention the most expensive part of inference. Training-free sparse attention is the cheapest fix: keep the pretrained weights, compute only a subset of key-value blocks per step, skip the rest. Existing methods fall down on two things.
First, the routing that picks blocks is rigid and costly. top-k fixes the same budget for every query; top-p keeps blocks until cumulative probability hits a threshold, which is dynamic but unbalanced. Both have to compute and materialize a full proxy-score map (a table of how important each block is) into HBM before sparse attention even starts, and at long sequences that round trip is real overhead. Second, blocks that fail selection are dropped outright, so aggressive sparsity costs accuracy.
Sol-Attn (Sparsifying online attention) starts from an observation: in pretrained video models, the pre-softmax proxy scores each query assigns to key blocks are approximately Gaussian. If they are Gaussian, you do not need to rank every block; a threshold will do.
The threshold is per-query. For query i, take the mean mui and standard deviation sigmai of its proxy-score row and set taui = mui + beta sigmai, where beta is one global standardized offset that controls overall sparsity. Each query then gets an adaptive budget (some need more blocks, some fewer), but the overall density stays stable, unlike top-p's swings.
The hard part is fusing that threshold test into a single online-softmax kernel pass. Three things collapse into one streaming operator:
Fusing routing, sparse computation, and correction into one streaming operator is the conceptual move here: routing is no longer a separate prerequisite, and unselected blocks are approximated rather than dropped.
The headline arena is text-to-video. On LTX 2.3, dense attention (FlashAttention-3) scores 75.90 on VBench; Sol-Attn reaches 76.13, slightly above dense, at a 2.02x end-to-end speedup. Against other sparse methods:
| Method | VBench (higher is better) | End-to-end speedup |
| FlashAttention-3 (dense) | 75.90 | 1.00x |
| PISA | 76.03 | 1.86x |
| Sparse-VideoGen2 | 75.22 | 1.85x |
| XAttention | 73.90 | 1.61x |
| Sol-Attn | 76.13 | 2.02x |
For video editing (Bernini) Sol-Attn hits 2.34x against Sparse-VideoGen2's 2.04x, and for one-minute video refinement 3.04x against 2.08x. At the kernel level on an H100, high-sparsity configs run 11.5x to 32.7x faster than FlashAttention-3, while Sparse-VideoGen2's routing structures need about 8x more memory.
The most quotable number comes from stacking Sol-Attn into NVIDIA's own Sol-Engine inference framework: kernel fusion, diffusion-step caching, and Sol-Attn together reach 3.48x on Wan2.1-14B and 5.08x on HunyuanVideo on a B200. Standalone, Sol-Attn is 2.1x to 3.0x.
This is a drop-in, training-free kernel. Teams running Wan, Hunyuan, or LTX can swap it in directly. Video generation inference is expensive, so a two- to three-fold speedup at no quality cost and no retraining has clear value. The design move (fuse routing into the softmax pass, approximate instead of drop) is not specific to video and transfers to other long-sequence diffusion models.
The authors name three: the B200 kernel does not yet fully exploit Blackwell; it supports forward inference only, with no backward pass, so it cannot be used in training; and evaluation covers bidirectional diffusion generation but not autoregressive video generation. One more caveat: nearly every comparison is against other training-free sparse methods, not against approaches that need light fine-tuning, so the lead is within the training-free lane rather than overall.