Running Qwen 27B on 2x P40s: parallel agents wreck KV cache, seeking a serial agent harness
Jumpy-Operation-4615 · reddit · 2026-09-14
The author runs Qwen 27B locally on 2x NVIDIA P40s, hitting 45 tg/s and 450 prefill (dropping to 120 prefill at 150K+ ctx). Problem: running multiple opencode agents (orchestrator, fixer, oracle) concurrently destroys each other's KV cache and prefix caching collapses. They ask whether a serial harness exists — supervisor delegates, sleeps, and wakes only after a subagent finishes and writes a full handoff — keeping one process active at all times to preserve prefix caching.
More from coding & agent
- Anthropic quietly ships server-side rule banning game rewrites in Claude Code — petrusenko_max · 2026-10-31
- Bootstrapping competitive edge with frontier tokens — curious_vii · 2026-10-31
- Element Forge: open-source editable AI VFX for real footage in ComfyUI — zitroffell · 2026-10-30
- Gym-Anything Adds Audit Agents That Try to Break Simulations With Legal Robot Actions — wellecks · 2026-10-30
- durable-object-migrator: zero-downtime exports for Cloudflare Durable Objects — samgoodwin89 · 2026-10-30
- This 5-question interview prompt makes Grok design your first two custom bots in minutes — nateliason · 2026-10-30