Slow prompt processing? Reddit user proposes pre-submitting agent overhead to warm the KV cache
butterfly_labs · reddit · 2026-09-20
A developer running large models (DS4 Flash, Qwen3.8 Flash) locally on an M2 Ultra with slow prompt processing reports that the fixed overhead an agent harness passes — system prompt, tool definitions, skills — takes 2-3 minutes to process before the first turn, after which the context is cached and subsequent turns get much faster.
Since this overhead is identical for a given project, they propose having the harness (they use Opencode) submit it as soon as the project opens, letting the machine warm the cache while the user types their first prompt. The post asks whether this pre-caching approach is viable.
More from coding & agent
- AI Capex vs AI Opex: Stop Paying Agents to Re-solve the Same Problem — njyx · 2026-09-20
- Gemini CLI PR fixes folder trust not persisting when running in podman/docker sandbox — 21vedansh · 2026-09-20
- Dev builds paged LLM inference engine for Apple Silicon, cuts attention memory 128x — Accomplished_Row1433 · 2026-09-20
- Reddit debate: We're building AI agents at the wrong scale — aim for persistent AI organizations — Druss_ · 2026-09-20
- Building long-term memory for agents: a philosophy, not a product — GrungeWerX · 2026-09-20
- AI engineer: without proper evals you're flying blind when building — antoine_chaffin · 2026-09-20