TokenSpeed MLA 2 hits NVIDIA Vera Rubin: up to 3.44x faster decode vs B200
zhyncs42 · x · 2026-10-16
LightSeek Foundation released TokenSpeed MLA 2, an optimized multi-head latent attention (MLA) decode kernel for NVIDIA Vera Rubin, built for long-horizon agentic inference.
- Performance: In long-context benchmarks, MLA 2 on Vera Rubin delivers up to 3.44× kernel speedups over MLA 1 on B200 and up to 2.84× over MLA 1 on GB200, combining hardware and software gains.
- Technical details: Leverages Vera Rubin's K=64-byte FP8 MMA instruction (2× arithmetic throughput per SM per clock), larger shared memory and TMEM; builds on a FlashInfer-derived pipeline adapted to Kimi K3's 96 query heads (H96) with Q4/Q8 configurations.
- Background: The original TokenSpeed MLA launched in May 2026 for Blackwell long-context agentic inference, beating the then-state-of-the-art TensorRT-LLM MLA baseline and serving Kimi K2.5 and later K3; the tokenspeed-mla package has passed 2 million monthly PyPI downloads.
More from Infra
- Intel's neural-compressor: SOTA low-bit LLM quantization with FP4/NVFP4 support — HaihaoShen · 2026-10-16
- Tencent squeezes 770B-parameter Hunyuan Hy4 Preview into 214 GiB at ~2.38 bits per weight — coolmum18 · 2026-10-16
- US vs Chinese hyperscalers: Americans fund and schmooze, Chinese demand your BoM and cap you at 7% margin — jwt0625 · 2026-10-16
- How LiteLLM routes one OpenAI-shaped API across every model provider — Mahmoud_Zalt · 2026-10-16
- OpenAI-Broadcom custom AI chip Jalapeño named a TIME Best Invention of 2026 — itsclivetime · 2026-10-16
- Developer live-runs continual pretraining of a Luganda LLM on Colab, 41% done with loss dropping fast — weskambale · 2026-10-16