VC-Attention: Training-Free Low-Bit Attention Speeds Video Diffusion Up to 3.6x
nunchux-inc · hf · 2026-09-17
For diffusion transformers, long spatiotemporal sequences make attention the dominant deployment cost, and low-bit quantization hits two walls: value outliers with no fixed channel/spatiotemporal structure dominate output error, and the high-precision softmax exponential becomes the longest pipeline stage on datacenter GPUs.
VC-Attention is a training-free framework pairing:
- V-Smooth: reorders value tokens via lightweight online clustering so tokens in a hardware block quantize well together; quantizes only the residual after subtracting the block mean, restoring the mean from the row sum online softmax already maintains
- ExpCast-FP8: maps log-domain scores directly to E4M3 probability codes with one fused multiply-add, eliminating the FP32 exponential and format conversion
Implemented for B200, B300, H200, RTX PRO 6000, and RTX 5090. Across Wan2.2, LongCat-Video, HunyuanVideo-1.5, and MiniMax-H3, it beats low-bit baselines on fidelity and speeds the attention kernel 1.46–1.59x over BF16 FlashAttention-4 on datacenter Blackwell/Hopper and 2.3–3.6x on workstation cards; end-to-end clip generation is 1.13–1.19x and 1.36–1.70x faster respectively.
Related event: VC-Attention: Training-Free Low-Bit Attention Speeds Up Inference on B200(3 posts)→
More from Infra
- Walmart's Training-Free Tri-Metric Router Cuts RAG OOM Failures to 0% on 16GB GPUs — _reachsumit · 2026-09-17
- Nebius raises on-demand GPU prices up to 21% starting Oct 1 — sudoraohacker · 2026-09-17
- Pinterest Details the Evolution of Its Billion-Scale Embedding Retrieval Models — AxSaucedo · 2026-09-17
- The egocentric data boom: Maxinsights has delivered 2M+ hours for robot training — 机器之心 · 2026-09-17
- Winbond to Buy Infineon's NOR Flash Business for $1.12B, Becoming World's Largest Maker — zephyr_z9 · 2026-09-17
- IonQ and ORNL demonstrate generative AI for quantum optimization — donutloop · 2026-09-17