FVAttn cuts video-generation load imbalance and speeds attention up 4.41×
Hao Liu · hf · 2026-07-23
FVAttn proposes a training-free sparse-attention system for video generation that tackles the load imbalance created by adaptive Top-p routing under multi-GPU sequence parallelism.
What it changes
- Uses a Top-p routing + Top-k safety floor frontend with video-aware block organization.
- Repairs sparse masks at runtime and moves heavy heads via P2P communication to shorten the critical path.
- Adds Slack-Aware Sparse Augmentation to fill leftover compute slack on less loaded ranks.
Reported results
- On step-distilled Wan2.2 I2V, the method reduces average load imbalance from 1.34 to 1.08.
- It delivers 4.41× attention speedup over FlashAttention.
- End-to-end, it achieves 2.02–2.11× DiT inference speedup with competitive video quality.
More from Multimodal
- FLUX.2 Klein Drifts Hard on Character Expressions While Free Gemini Holds Likeness — wacomlover · 2026-09-11
- Tencent Hunyuan releases AuK code and weights on GitHub with ComfyUI and fine-tuning support — aigclink · 2026-09-11
- Creator turns Bahamut vs Tiamat rivalry into an AI cinematic battle with Midjourney, GPT Image 2 and Seedance — azed_ai · 2026-09-11
- invideo launches AI agent-powered editor to automate repetitive editing tasks — azed_ai · 2026-09-11
- fable 5.1 recreates The Starry Night with 256,157 JavaScript brush strokes — cedric_chee · 2026-09-11
- GPT-6 Astra + Hyper3D Rodin MCP Generates 3D Assets in One Agent Flow — ahuja_priyank · 2026-09-11