FVAttn cuts video-generation load imbalance and speeds attention up 4.41×
Hao Liu · hf · 2026-07-23
FVAttn proposes a training-free sparse-attention system for video generation that tackles the load imbalance created by adaptive Top-p routing under multi-GPU sequence parallelism.
What it changes
- Uses a Top-p routing + Top-k safety floor frontend with video-aware block organization.
- Repairs sparse masks at runtime and moves heavy heads via P2P communication to shorten the critical path.
- Adds Slack-Aware Sparse Augmentation to fill leftover compute slack on less loaded ranks.
Reported results
- On step-distilled Wan2.2 I2V, the method reduces average load imbalance from 1.34 to 1.08.
- It delivers 4.41× attention speedup over FlashAttention.
- End-to-end, it achieves 2.02–2.11× DiT inference speedup with competitive video quality.
More from Multimodal
- Reddit users ask whether Hunyuan Image 3.0 Instruct is worth 170GB VRAM — dtdisapointingresult · 2026-07-23
- CoLT teaches multimodal models to think in latent steps and cuts inference 10.1x — jiqizhixin · 2026-07-23
- ComfyUI gets an open-source TTS and voice-cloning workflow — Goble4 · 2026-07-23
- AI-generated short drama apps now occupy 12 of the top 50 U.S. entertainment apps — deedydas · 2026-07-23
- VidBoards turns AI image and video comparisons into an open-source canvas app — shangTsungTeaMaster · 2026-07-23
- K3 reportedly beats GPT-5.6 and Fable 5 on several benchmarks — yihui_indie · 2026-07-23