Open-source prefill engine hits 3.2× vLLM throughput serving GLM-5.3-Flash on 4 B200s
bariskasikci · x · 2026-11-19
Kan Zhu released SSEngine-GLM5.3-Flash, an open-source prefill engine purpose-built for serving GLM-5.3-Flash on four B200 GPUs in coding-agent workloads. It combines pipeline parallelism, a tiered prefix cache spanning GPU memory, DRAM and SSD, and a shortest-remaining-prefill-first scheduler. Replaying real coding-agent traces (AgentX and TraceLab), it sustains 3.1× and 3.2× the request rate of every vLLM or SGLang setup tested. Quoting commentary notes the industry is already building bespoke serving stacks per model and chip, and argues prefill and decode each deserve their own serving stack.
More from coding & agent
- Netlify rebrands around agentic app building: describe, preview, deploy in one flow — alifcoder · 2026-11-19
- Solo dev cuts sandbox cold-start to 0.45s, sleeping VMs cost 1¢/month — emilahlback · 2026-11-19
- GKE sandbox metrics revealed: 0.45s startup, 1¢/month asleep, failures down from 1,081 to zero — emilahlback · 2026-11-19
- Team cuts agent sandbox cost 88% by moving to GKE, open-sources gke-sandboxes — emilahlback · 2026-11-19
- Google Antigravity shipped 150 releases in 90 days — a Googler's hands-on review finds it finally caught up — rseroter · 2026-11-19
- First Person Labs CEO: AI should connect the dots across months, not wait for prompts — a16z · 2026-11-19