DeepSeek-V4.1-Flash hits 5x agent throughput on vLLM via SWA bounded replay
vllm_project · x · 2026-10-08
vLLM highlights 37FlowAI's teardown of DeepSeek-V4.1-Flash: three weeks after launch, SWA bounded replay pushes agent-scenario throughput to 5x. Window KVs are not kept resident; on prefix hits only the last 128 tokens are recomputed, halving long-prompt prefill and cutting TTFT by 30%, with open-source new kernels contributing the rest. In agent serving, prefill is the biggest cost, so KV savings directly determine concurrency.
Related event: vLLM boosts DeepSeek-V4.1-Flash throughput up to 5.3x in three weeks(3 posts)→
More from coding & agent
- Musk amplifies Grok Bot feature list: own computer, email, and a team of bots — elonmusk · 2026-10-19
- PoA: An Experimental Network Where AI Agents Earn Coins by Writing Original Work — adamamcbride · 2026-10-19
- Next Token podcast: 'If AI gets something to 60 points, it's worth zero' — op7418 · 2026-10-19
- Running GLM on 4x DGX Spark locally to build a 3D Three.js portfolio site — 9r4n4y · 2026-10-19
- User claims Amazon spending dropped >80% after platform banned his AI shopping agent — zeroxjackson · 2026-10-19
- Anthropic engineer: 99% use Claude Code like Google search, 1% run self-learning agent swarms — msharmas · 2026-10-19