DeepSeek open-sources DeepInfer, its production inference engine; 6.3K stars in 8 hours
rohanpaul_ai · x · 2026-10-20
DeepSeek released DeepInfer, the inference engine behind its production services, under Apache 2.0 — it hit 6.3K+ GitHub stars in 8 hours.
- Scope: runs only DeepSeek-V4.1-Flash, on NVIDIA SM100 GPUs or Huawei Ascend 950 NPUs; offline inference only — no KV cache system, no prefill/decode disaggregation, no load balancing across replicas
- Speed-layer tech: DSpark drafts up to 5 tokens ahead with confidence-scheduled verification; SWA bounded replay recomputes only a 140-token suffix covering the sliding attention window; plus a zero-overhead async batch scheduler, batch-invariant inference, deterministic sampling, structured outputs
- Companion: deepseek-recipe converts Chat Completions / Responses / Messages requests into DeepSeek V4/V4.1 prompts matching the official API
Related event: DeepSeek Open-Sources DeepInfer Inference Framework(24 posts)→
More from Infra
- Chrome 155 ships JPEG XL: 30-50% smaller than JPEG with lossless JPEG transcode — banteg · 2026-10-20
- Polymarket: DeepSeek releases open-source tool to sharply cut AI inference compute costs — Polymarket · 2026-10-20
- Together, Fireworks, Prime Intellect and Modal are all converging on inference + RL training — NielsRogge · 2026-10-20
- NVIDIA GTC kicks off in Berlin with Jensen Huang keynote set for tomorrow — NVIDIAAI · 2026-10-20
- Hyperdrives CEO interview: cheaper power-dense electric motors via cooling breakthrough — MarvinTBaumann · 2026-10-20
- MiniMax H3 video generation runs on a single RTX 3060 12GB via GGUF quantization — Ok-Inevitable8391 · 2026-10-20