vLLM x AgentX: Full-Stack Agentic Serving Optimization Cuts Cost 106x vs Opus 5

The vLLM team (in collaboration with Inferact, per one post) published the blog "vLLM x AgentX: Optimizing for Real-World Agentic Serving," based on SemiAnalysis's public agentic benchmark AgentX, presenting full-stack inference optimizations for real-world agent workloads. The team concludes that optimized vLLM serving achieves up to a 106x throughput-cost advantage over an Opus 5 API-based solution — i.e., DeepSeek V4 Pro inference costs just 1/106 of it. The team also notes that agent traffic is now a major traffic source for vLLM.

Confirmed

• Traffic profile from real coding-agent traces: median 43 turns per session, median input 142K tokens vs only 444 output tokens, prefix cache hit rate over 96%, and 44% of sessions fork sub-agents
• KV cache management: packed KV layout for DeepSeek V4 saves 10% KV memory, condensing 92 tensors per block to 1
• Kimi K3 decode throughput improved 2.7x
• Serving cost up to 106x lower than the Opus 5 API option
• Multi-turn sessions, long contexts, and heavy prefix reuse impose new demands on KV cache management and scheduling; agent traffic stresses every layer of the serving stack, requiring coordinated optimization across architecture, framework, and runtime

Why it matters

Agent workloads — long inputs, short outputs, heavy prefix reuse, and multi-turn forking — differ sharply from traditional chat/completion loads, directly shaping KV cache design and scheduling in inference engines. By publishing measured results on the public AgentX benchmark, vLLM offers the community a reproducible reference and reinforces its position as the go-to serving stack for the agent era.

2026-09-09 ~ 2026-09-09 · 7 related posts

• [source] vLLM details full-stack optimizations for real-world agentic serving on AgentX benchmark — vllm_project · 2026-09-09
• vLLM's agentic serving deep-dive: DeepSeek V4 Pro hits 83K tok/GPU-s at 106x lower cost than Opus 5 — vllm_project · 2026-09-09
• vLLM: agent sessions median 43 turns, 142K-token inputs vs 444-token outputs — vllm_project · 2026-09-09
• [source] vLLM agentic serving details: packed KV saves 10% memory, Kimi K3 decode throughput up 2.7x — vllm_project · 2026-09-09
• [source] vLLM on AgentX: up to 106× serving-cost advantage over Opus 5 API pricing — vllm_project · 2026-09-09
• vLLM: DeepSeek V4 Pro serving can be 106x cheaper than Opus 5 on AgentX workloads — woosuk_k · 2026-09-09
• vLLM x AgentX: Full-Stack Optimizations for Real-World Agentic Serving — jfiance · 2026-09-09