vLLM Unveils AgentX Benchmark: Up to 106x Cost Advantage for Real-World Agentic Serving
The vLLM project, together with SemiAnalysis (another post mentions Inferact), published a long-form piece, "vLLM x AgentX: Optimizing for Real-World Agentic Serving," based on the public agentic benchmark AgentX, systematically covering full-stack inference optimizations for real-world agent workloads. The team's conclusion: compared to solutions based on the Opus 5 API, the optimized vLLM serving throughput delivers a cost advantage of up to 106x.
Confirmed
- Traffic profiles come from real coding agent traces: median session of 43 turns, median input of 142K tokens with only 444 output tokens, prefix cache hit rate above 96%, and 44% of sessions fork sub-agents
- KV cache management optimizations: packed KV layout for DeepSeek V4 saves roughly 10% KV memory and consolidates 92 tensors per block down to 1
- Kimi K3 decode throughput improved 2.7x
- Serving cost is up to 106x lower than the Opus 5 API solution
- Officials note that agent traffic has become vLLM's dominant traffic source, and multi-turn sessions, long contexts, and heavy prefix reuse impose new demands on KV cache management and scheduling
Why it matters
Agent workloads — long inputs, short outputs, heavy prefix reuse, multi-turn forking — differ markedly from traditional chat/completion loads, directly shaping inference engines' KV cache design and scheduling strategies. By publishing results on the public AgentX benchmark, vLLM offers the community a reproducible reference and reinforces its position as the go-to inference stack for the agent era
2026-09-09 ~ 2026-09-09 · 5 related posts
Primary sources
- vLLM details full-stack optimizations for real-world agentic serving on AgentX benchmark — vllm_project ·
- vLLM: agent sessions median 43 turns, 142K-token inputs vs 444-token outputs — vllm_project ·
- vLLM's agentic serving deep-dive: DeepSeek V4 Pro hits 83K tok/GPU-s at 106x lower cost than Opus 5 — vllm_project ·
- [source] vLLM details full-stack optimizations for real-world agentic serving on AgentX benchmark — vllm_project · 2026-09-09
- [source] vLLM's agentic serving deep-dive: DeepSeek V4 Pro hits 83K tok/GPU-s at 106x lower cost than Opus 5 — vllm_project · 2026-09-09
- [source] vLLM: agent sessions median 43 turns, 142K-token inputs vs 444-token outputs — vllm_project · 2026-09-09
- vLLM agentic serving details: packed KV saves 10% memory, Kimi K3 decode throughput up 2.7x — vllm_project · 2026-09-09
- vLLM on AgentX: up to 106× serving-cost advantage over Opus 5 API pricing — vllm_project · 2026-09-09