vLLM's agentic-api: stateful continuation ends 200K-token resends for agent harnesses
AccBalanced · x · 2026-10-19
EmbeddedLLM highlights a key agent inefficiency: after squeezing every microsecond out of kernels, harnesses still resend 200K tokens just to say "continue." Their new open-source vllm-project/agentic-api adds stateful continuation, smaller requests, lower latency, MCP support, server tools, and web search for vLLM, working with Codex, Claude Code, and other harnesses. Ray Summit slides show its Agentic API keeps client request size constant across turn depths, while direct vLLM requests balloon to 400K bytes.
More from coding & agent
- BELIEFWEAVER: belief updates and world modeling for long-horizon agent failure recovery — IntelligenceLab · 2026-10-19
- DeepMind's XYEval: one bad user hint drops agent success from 67.4% to 36% — i_dg23 · 2026-10-19
- REA: open-source MCP tool lets AI agents reverse engineer any app down to assembly — Aiden_Tech_Ai · 2026-10-19
- Pure three.js + Claude/Codex Yields Fully Procedural Game Characters — TinfoilTricorn · 2026-10-19
- Open-source Kapitsa: a Metal Engine for Apple Silicon Pushing Qwen 27B to 63 tok/s on M4 Pro — DotaMate · 2026-10-19
- One CLI Beats One MCP Tool per API: an Agent Tooling Workflow and a 7-Day OAuth Trap — tariqosmani · 2026-10-19