SparkDiffusion from PKU/Tsinghua/Alibaba hits 265x video generation speedup on a single RTX 5090
机器之心 · wechat · 2026-09-28
Researchers from Peking University, Tsinghua, and Alibaba released SparkDiffusion — the first unified open-source framework combining sparse attention, few-step distillation, and FP8 quantization to accelerate DiT video generation, with weights and full training code released.
- Key finding, the "High-Sparsity Trap": at 97% sparsity, training loss keeps dropping while video quality collapses (broken figures, temporal chaos), and longer training can't fix it. Oracle intervention experiments show the root cause: structural errors in high-noise steps get amplified along the denoising trajectory.
- Solution: Terminal-Aligned Supervision constrains what each step ultimately generates; RoLA sparse attention (block-sparse plus low-rank linear branch restoring global context) and CrossDistill trajectory-mixed distillation (PCM at high noise for diversity, DMD at low noise to fix endpoint errors) yield a 3-step CFG-free student, plus FP8 quantization.
- Results: 265x speedup on a single RTX 5090 (720P-14B, 5s video in 18s); 220x on H100 with latency down to 8s; VBench-2.0 only drops from 60.2 to 59.77.
- Why it matters: gains scale with resolution (O(L²) attention), bringing high-quality video generation to consumer GPUs; supports Wan2.1/2.2 T2V and I2V, with extensions to autoregressive and omni-modal generation planned.
More from Multimodal
- User Shares Coral Reef Creation Labeled 'Qwen 4 Max (xhigh)', an Unannounced Model Name — intellectronica · 2026-11-16
- Hand-editing 0.2% of weights plus 100 CPU-minutes turns an English TTS model into an 18MB Sanskrit one — kastnerkyle · 2026-11-16
- Zyphra open-sources textnorm tool, lifting TTS structured-data accuracy by up to 22 points — kastnerkyle · 2026-11-16
- Testing Qwen Image 2.1 and Minimax H3 on anime-to-photo conversion and film photo restoration — Ant_6431 · 2026-11-16
- Second take of a 'Cubee run' video generated with Minimax H3 — JahJedi · 2026-11-16
- Non-artist builds 14-chapter hand-painted storybook game with Fable 5.2 in 4 hours — jradoff · 2026-11-16