VC-Attention: Training-Free Low-Bit Attention Speeds Up Video Generation
Nunchux AI, together with researchers from MIT, CMU, UC Berkeley, Stanford and NVIDIA, released VC-Attention (arXiv:2609.15810), a training-free, accurate, low-bit attention method targeting diffusion Transformer video generation. It addresses two bottlenecks of low-bit quantization: Value outliers with no clear channel/spatial-temporal structure (the main source of output error) and the high-precision softmax exponent computation (the longest pipeline stage on data-center GPUs).
Confirmed
- On MiniMax-H3 video generation (243-frame long sequences), attention is 1.6x faster on B200, with additional speedups on B300 (exact figure not given).
- At 1344x768 resolution, attention computation is about 1.6x faster, and up to 3.58x on RTX 5090.
- It outperforms FlashAttention-4 and SageAttention2.
- The method requires no retraining and can be applied directly to existing models.
Why it matters
- The bottleneck of fast video generation is shifting from raw compute to attention and quantization details; VC-Attention shows that engineering Value-outlier handling and softmax can yield multi-fold attention speedups without retraining, directly reducing video-generation deployment costs.
- Accounts linked to the authors, including Song Han's group at MIT, and the official Nunchux AI account actively promoted the work.
2026-09-17 ~ 2026-09-18 · 5 related posts
Primary sources
- VC-Attention: training-free low-bit attention hits 1.9x on B200, beating FlashAttention-4 — xiuyu_l · 2026-09-17
- [source] VC-Attention: Training-Free Low-Bit Attention Speeds Video Diffusion Up to 3.6x — nunchux-inc · 2026-09-17
- New VC-Attention Paper Reports up to 3.58x Attention Speedup on RTX 5090 with MiniMax H3 — robomar_ai_art · 2026-09-18
2 near-duplicate retellings: songhan_mit · junyanz89