FVAttn cuts video-generation load imbalance and speeds attention up 4.41×
Hao Liu · hf · 2026-07-23
FVAttn proposes a training-free sparse-attention system for video generation that tackles the load imbalance created by adaptive Top-p routing under multi-GPU sequence parallelism.
What it changes
- Uses a Top-p routing + Top-k safety floor frontend with video-aware block organization.
- Repairs sparse masks at runtime and moves heavy heads via P2P communication to shorten the critical path.
- Adds Slack-Aware Sparse Augmentation to fill leftover compute slack on less loaded ranks.
Reported results
- On step-distilled Wan2.2 I2V, the method reduces average load imbalance from 1.34 to 1.08.
- It delivers 4.41× attention speedup over FlashAttention.
- End-to-end, it achieves 2.02–2.11× DiT inference speedup with competitive video quality.
More from Multimodal
- Butter.video wins over skeptical creator with hundreds of tasteful timeline 'blocks' for video effects — beechinour · 2026-10-27
- AI Video Pits Brock Lesnar Against Duncan the Tall in a Toy Showdown — KloneStudios · 2026-10-27
- $50, 2-hour Kurzgesagt-style AI video reignites debate over intent in AI creation — motionbynick · 2026-10-27
- Fake BTS of a dam disaster made with Dreamina Seedance for 29 cents — Standard_Spell5873 · 2026-10-27
- Redditor struggles with outfit swap: Qwen Image 2.1 breaks body proportions — Existing-Box-7866 · 2026-10-27
- Kling 4.0 lands on Deevid AI: 30-second videos, 10 keyframes, 15 multimodal references — nikola_mr64990 · 2026-10-27