SGLang ships Day-0 support for Kimi K3.1 Pro with sparse attention, up to 33.7K tok/s per GPU
ying11231 · x · 2026-11-07
SGLang announced Day-0 support for Kimi K3.1 Pro, which replaces dense MLA with sparse attention (DSA) for more efficient serving: up to 33.7K tok/s per GPU (AgentX, GB300), 450 tok/s single-user with DFlash2 + MegaKernel, across NVIDIA Vera Rubin/Blackwell/Hopper, AMD, TPU, Trainium and PPU. Verda Inference, built on SGLang and tuned by core contributors, serves the model from day 0.
Related event: Kimi K3.1 Pro Gains Day-0 Support from You.com and SGLang(2 posts)→
More from Infra
- JITEngine synthesizes custom LLM inference engines for Apple Silicon, claims up to 2.7x speedup — AccBalanced · 2026-11-07
- Dev predicts 2027's local inference boom: integrated hardware + OS + inference stack, not just GPUs — vincent_koc · 2026-11-07
- vLLM Office Hours Hits 59 Sessions: 76 Speakers from 24 Organizations and Counting — vllm_project · 2026-11-07
- Dev runs 284B-parameter DeepSeek-V4-Flash on CPU only, squeezed into 64GB DDR5 RAM — solyarisoftware · 2026-11-07
- Omnigraph: open-source graph database with git-style branching for multi-agent shared memory — solyarisoftware · 2026-11-07
- Solo dev ships all-Rust bootable OS that runs local LLMs up to 25% faster than llama.cpp — W61k3r · 2026-11-07