FVAttn cuts video-generation load imbalance and speeds attention up 4.41×
Hao Liu · hf · 2026-07-23
FVAttn proposes a training-free sparse-attention system for video generation that tackles the load imbalance created by adaptive Top-p routing under multi-GPU sequence parallelism.
What it changes
- Uses a Top-p routing + Top-k safety floor frontend with video-aware block organization.
- Repairs sparse masks at runtime and moves heavy heads via P2P communication to shorten the critical path.
- Adds Slack-Aware Sparse Augmentation to fill leftover compute slack on less loaded ranks.
Reported results
- On step-distilled Wan2.2 I2V, the method reduces average load imbalance from 1.34 to 1.08.
- It delivers 4.41× attention speedup over FlashAttention.
- End-to-end, it achieves 2.02–2.11× DiT inference speedup with competitive video quality.
More from Multimodal
- Dense instance segmentation via Muse Wonder + SAM 3.1 on Meta's Model API — nikhilaravi · 2026-10-26
- LichtFeld Studio v0.5.4.3 ships community-built SteadySplats, POPSpa and Voxelization — janusch_patas · 2026-10-26
- One prompt to a 2-3 min narrated short: hands-on test of Antigravity video generation — AI_Andrew · 2026-10-26
- Creator makes 10,564-frame animated short entirely with Gemini toolchain — AI_Andrew · 2026-10-26
- FLUX.3 Klein step-distilled to 4-8 steps with CFG baked in — Fit-Structure-6489 · 2026-10-26
- ostris pre-training "Primal VAE": 1-channel latents match modern 8x VAEs via patching — ostrisai · 2026-10-26