OSDI Paper: vLLM and Others Unsuitable for Long-Running Coding Agents
AccBalanced · x · 2026-07-19
At USENIX OSDI, researchers from UW and UVA presented a paper on tuning vLLM for coding agents.
- Background: They collected the most comprehensive Claude Code OS-level tracing dataset from their student labs.
- Core Finding: Existing serving frameworks like vLLM and SGLang cannot directly handle long-running agent workloads.
- Technical Detail: If KV cache is constrained, more CPU scheduling overhead is needed to predict cache reuse in long coding sessions.
More from coding & agent
- 250 Builders Hack With Codex at Berlin Community Hackathon Ahead of OpenAI DevDay — paw_lean · 2026-10-26
- Ben Bajarin: Frontier AI is becoming the enterprise cyber-defense stack — BenBajarin · 2026-10-26
- Building AI bots on raw APIs? Devs say models are 'dumb without the harness' — madeofAjala · 2026-10-26
- HarnessMM cuts agent tokens per task by up to 88% while raising success rates — omarsar0 · 2026-10-26
- Microsoft demos intent-classifying agents built with Copilot App and Foundry, deployable across Teams and Outlook — marlene_zw · 2026-10-26
- Looking for a wearable mic with best battery life that feeds recordings to personal agents via MCP — altryne · 2026-10-26