vLLM matches DeepInfer on DeepSeek-V4.1-Flash across batch sizes 8–256
vllm_project · x · 2026-10-25
vLLM officially reports that it matches DeepInfer, DeepSeek's own engine for DeepSeek-V4.1-Flash, on DeepSeek's decode configuration across batch sizes 8 to 256 — running DEP32 against DeepInfer's EP64DP64. DeepInfer was originally built on vLLM; by integrating DeepSeek's open-source kernels and extensively optimizing prefill and decode, vLLM now tracks DeepInfer's performance curves. The attached chart compares decode and prefill throughput-latency for both engines.
Related event: vLLM Matches DeepSeek's In-House DeepInfer Engine Performance(2 posts)→
More from Infra
- Jev ecosystem explodes: 5 open-source apps from a 7.1s flight-search agent to SQL-in-English — udmrzn · 2026-10-25
- AWS Interconnect goes multicloud: GA with Google Cloud, OCI added, Azure in preview — DavidLinthicum · 2026-10-25
- CMU professor jokes about funding his lab by reselling B300 compute as DeepSeek tokens — niloofar_mire · 2026-10-25
- Hyperscaler depreciation to hit $590B by 2029, likely exceeding combined operating profits — GaryMarcus · 2026-10-25
- vLLM hits 954K tokens/s per GPU serving DeepSeek V4.1-Flash on GB300 NVL72 rack — vllm_project · 2026-10-25
- UK per-capita electricity now below 1970s levels, just as AI demands power — NinaDSchick · 2026-10-25