Open-source prefill engine hits 3.2× vLLM throughput serving GLM-5.3-Flash on 4 B200s

bariskasikci · x · 2026-11-19

Kan Zhu released SSEngine-GLM5.3-Flash, an open-source prefill engine purpose-built for serving GLM-5.3-Flash on four B200 GPUs in coding-agent workloads. It combines pipeline parallelism, a tiered prefix cache spanning GPU memory, DRAM and SSD, and a shortest-remaining-prefill-first scheduler. Replaying real coding-agent traces (AgentX and TraceLab), it sustains 3.1× and 3.2× the request rate of every vLLM or SGLang setup tested. Quoting commentary notes the industry is already building bespoke serving stacks per model and chip, and argues prefill and decode each deserve their own serving stack.

Original post →

More from coding & agent

coding & agent channel →