VC-Attention: training-free low-bit attention speeds up MiniMax-H3 1.6x on B200
junyanz89 · x · 2026-09-18
Nunchux AI, with researchers from MIT, CMU, UC Berkeley, Stanford and NVIDIA, introduced VC-Attention, a training-free low-bit attention acceleration method. On MiniMax-H3 it delivers 1.6x speedup on B200 and 1.5x on B300 over FlashAttention-4, with better fidelity than SageAttention-2, and composes with sparse attention methods.
Key innovations:
- V-Smooth: reduces value quantization error
- ExpCast-FP8: accelerates softmax via approximation
The proprietary Nunchux Attention extension pushes speedups to 1.9x (B200) and 1.8x (B300). MiniMax shared the news alongside a visual guide to three paths for faster video attention, and suggested combining low-bit compute with sparse methods like Sol-Attn as future work. Blog and technical report included.
Related event: VC-Attention: Training-Free Low-Bit Attention Speeds Up Video Generation(5 posts)→
More from Infra
- Sam Altman: AI query energy cost fell 1,000x-100,000x in 3-4 years — rohanpaul_ai · 2026-11-02
- Marvell and Lumentum to merge, targeting $550B AI connectivity market — BenBajarin · 2026-11-02
- Samsung chairman visits Yokohama advanced packaging lab, pledging ¥40B investment over five years — pstAsiatech · 2026-11-02
- Sam Altman meets Samsung executives in Korea to deepen memory chip and AI infrastructure cooperation — pstAsiatech · 2026-11-02
- Simulating 200,000 months: can your home AI rig pay rent at 2026 GPU prices? — LeviTurk · 2026-11-02
- Indian AI infra firm AM Intelligence seeks up to $3B to buy Nvidia's next-gen chips — pstAsiatech · 2026-11-02