Keeping vLLM's prefix cache warm between agent turns: an engineering guide
bolts98 · reddit · 2026-09-17
An engineering post explains how to keep vLLM's prefix cache warm across agent turns so KV computation isn't redone each time. It covers why cache hits are commonly lost (requests routed to different workers, reordered context) and concrete fixes: keeping prefixes stable, session affinity, and cache warming—cutting latency and compute for multi-turn agent workloads.
More from coding & agent
- 90% of enterprise agent pilots fail in production; AGI House hosts one-day sprint on delivery — agihouse_org · 2026-09-17
- Codex builds an iOS app and auto-submits it to the App Store end to end — ezshine · 2026-09-17
- Hermes Agent plugin ecosystem grows as user ditches OS to live inside the desktop app — max_paperclips · 2026-09-17
- Computer use in Muse: a vision-plus-actions loop that generalizes to any website — armand_ruiz · 2026-09-17
- DuckDB extension uses Jev to classify CSV rows in ~10s per 1k rows — josh_wills · 2026-09-17
- omitClaudeMd cuts Claude Code subagent context by ~88% across 10 agents — This_Cell_1829 · 2026-09-17