LMSYS XM-Sparse cuts decode time up to 63.6% with HiSparse plus X-Mem KV offload
xiaosun86 · x · 2026-11-06
LMSYS announced XM-Sparse, combining SGLang HiSparse sparse attention with Netpreme's X-Mem memory system, which fetches offloaded KV over a high-bandwidth interconnect faster than host DRAM over PCIe while leaving room for larger batches.
- Faster decode: per-token time 49.25%/63.58% lower than host DRAM at 20%/40% KV miss rates (16K context, batch 192/GPU)
- Higher throughput: 2.09–3.04× host DRAM across 16K–1M contexts at 20% misses
Currently a pilot VM evaluation; X-Mem silicon is forthcoming.
More from Infra
- Foundry Local beats llama.cpp by 35-135% on long-context decoding for on-device agentic coding — pmttyji · 2026-11-06
- Texas data center debate: a look at Dickens County's Helios campus — builderjaydub · 2026-11-06
- Polymarket poll: 56% of voters say AI and data centers shaped their midterm vote — KyeGomezB · 2026-11-06
- New attention kernels meet real models: where interpretability overlaps systems research — QuentinAnthon15 · 2026-11-06
- MoE All-to-All Comms Eat 60% of Training Steps; Correlated Placement Cuts 58% of Dispatched Rows — QuentinAnthon15 · 2026-11-06
- Texas data center bill would let all permitted projects finish, just with stricter consumer rules — VraserX · 2026-11-06