SGLang boosts DeepSeek V4.1 Flash throughput 67% to 1.09M tok/s per GPU in 4 days
BanghuaZ · x · 2026-10-31
The SGLang team reports DeepSeek V4.1 Flash peak total throughput (cache + new input + output) on AgentX jumped from 654K to 1.09M tok/s per GPU in just four days. For ultra-low latency, MegaKernel measured 943 tok/s/user at concurrency 1. The team says more optimizations are coming — and the post hints at the existence of a new DeepSeek V4.1 Flash model.
More from Infra
- Claude Code background agents burn ~$4.80/hour just writing status lines, dev calculates — arpit_bhayani · 2026-10-31
- Two weeks of agent logs: INT4 quantization triples Qwen 27B's reasoning loops at long context — Old_Ad_6033 · 2026-10-31
- YOLOv2 Face Detection Hits 30fps on a K210 Chip via Custom C++ Firmware — alexcovo_eth · 2026-10-31
- Playable Environments: Millisecond Cloud Branching as the Next Test Bed for Agents — matei_zaharia · 2026-10-31
- Apple silicon is so overpowered only local LLM runners and 8K editors notice — vivekhaldar · 2026-10-31
- Ming-TTS inference ported to Zig/MLX hits 18x speedup on M4/M5 chips — s_busso · 2026-10-31