TokenSpeed MLA 2 ships for Vera Rubin: up to 3.44× faster Kimi K3 decode
zhyncs42 · x · 2026-10-16
LightSeek's TokenSpeed team released MLA 2, an MLA decode kernel optimized for NVIDIA Vera Rubin, succeeding the Blackwell-era MLA 1 (echoing the FlashAttention 1→4 progression). On long-context benchmarks it delivers up to 3.44× decode speedup for Kimi K3 vs. MLA 1 on B200 and up to 2.84× on GB200, combining hardware and software gains. Key techniques: exploiting Vera Rubin's K=64-byte FP8 MMA instruction (2× per-SM throughput vs. K=32), larger shared memory/TMEM for wider tiles, a FlashInfer-derived pipeline adapted to Kimi K3's 96 query heads with Q4/Q8 workloads, and QK/PV pipeline overlap. MLA 1 launched May 2026, beat the then-SOTA TensorRT-LLM MLA baseline, and served Kimi K2.5/K3; tokenspeed-mla now exceeds 2M monthly PyPI downloads. Built with early access hardware from NVIDIA AI.
More from Infra
- First AA-AgentPerf results: NVIDIA Vera Rubin NVL72 delivers up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
- Vera Rubin NVL72 first benchmarks: up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
- NVIDIA Vera Rubin NVL72 hits up to 5.7x GB300 throughput per MW on agent benchmark — ArtificialAnlys · 2026-10-16
- 14 open-source LLM observability tools to know in 2026, from Langfuse to Laminar — TheTuringPost · 2026-10-16
- Local AI is a routing problem, not a privacy feature: where each task should run — bigdata · 2026-10-16
- Moving agent work from GPUs to CPUs for durable, predictable actions — cramforce · 2026-10-16