TokenSpeed MLA 2 hits Vera Rubin: up to 3.44× faster MLA decode for Kimi K3
zhyncs42 · x · 2026-10-16
LightSeek Foundation released TokenSpeed MLA 2, an MLA decode kernel optimized for NVIDIA Vera Rubin, targeting Kimi K3 long-context agentic inference (96 query heads, Q4/Q8).
- Up to 3.44× kernel speedup vs MLA 1 on B200, and 2.84× vs MLA 1 on GB200 in long-context benchmarks (combined hardware + software gains)
- MLA 1 shipped May 2026 for Blackwell, beat the then-SOTA TensorRT-LLM MLA baseline, and has served Kimi K2.5 and K3
- The tokenspeed-mla package now exceeds 2M monthly PyPI downloads
- Built on a FlashInfer-derived pipeline, leveraging Vera Rubin's FP8 K=64B MMA (2× throughput per SM per clock), larger shared memory/TMEM, and CTA-pair overlap of QK and PV phases
More from Infra
- Intel's neural-compressor: SOTA low-bit LLM quantization with FP4/NVFP4 support — HaihaoShen · 2026-10-16
- Tencent squeezes 770B-parameter Hunyuan Hy4 Preview into 214 GiB at ~2.38 bits per weight — coolmum18 · 2026-10-16
- TokenSpeed MLA 2 hits NVIDIA Vera Rubin: up to 3.44x faster decode vs B200 — zhyncs42 · 2026-10-16
- US vs Chinese hyperscalers: Americans fund and schmooze, Chinese demand your BoM and cap you at 7% margin — jwt0625 · 2026-10-16
- How LiteLLM routes one OpenAI-shaped API across every model provider — Mahmoud_Zalt · 2026-10-16
- OpenAI-Broadcom custom AI chip Jalapeño named a TIME Best Invention of 2026 — itsclivetime · 2026-10-16