VC-Attention: Training-Free Low-Bit Attention Speeds Video Diffusion Up to 3.6x
nunchux-inc · hf · 2026-09-17
For diffusion transformers, long spatiotemporal sequences make attention the dominant deployment cost, and low-bit quantization hits two walls: value outliers with no fixed channel/spatiotemporal structure dominate output error, and the high-precision softmax exponential becomes the longest pipeline stage on datacenter GPUs.
VC-Attention is a training-free framework pairing:
- V-Smooth: reorders value tokens via lightweight online clustering so tokens in a hardware block quantize well together; quantizes only the residual after subtracting the block mean, restoring the mean from the row sum online softmax already maintains
- ExpCast-FP8: maps log-domain scores directly to E4M3 probability codes with one fused multiply-add, eliminating the FP32 exponential and format conversion
Implemented for B200, B300, H200, RTX PRO 6000, and RTX 5090. Across Wan2.2, LongCat-Video, HunyuanVideo-1.5, and MiniMax-H3, it beats low-bit baselines on fidelity and speeds the attention kernel 1.46–1.59x over BF16 FlashAttention-4 on datacenter Blackwell/Hopper and 2.3–3.6x on workstation cards; end-to-end clip generation is 1.13–1.19x and 1.36–1.70x faster respectively.
Related event: VC-Attention: Training-Free Low-Bit Attention Speeds Up Video Generation(5 posts)→
More from Infra
- Marvell-Lumentum deal signals what it takes to compete with Broadcom — BenBajarin · 2026-11-02
- Sam Altman: AI query energy cost fell 1,000x-100,000x in 3-4 years — rohanpaul_ai · 2026-11-02
- Marvell and Lumentum to merge, targeting $550B AI connectivity market — BenBajarin · 2026-11-02
- Samsung chairman visits Yokohama advanced packaging lab, pledging ¥40B investment over five years — pstAsiatech · 2026-11-02
- Sam Altman meets Samsung executives in Korea to deepen memory chip and AI infrastructure cooperation — pstAsiatech · 2026-11-02
- Simulating 200,000 months: can your home AI rig pay rent at 2026 GPU prices? — LeviTurk · 2026-11-02