Running Qwen 27B on 2x P40s: parallel agents wreck KV cache, seeking a serial agent harness
Jumpy-Operation-4615 · reddit · 2026-09-14
The author runs Qwen 27B locally on 2x NVIDIA P40s, hitting 45 tg/s and 450 prefill (dropping to 120 prefill at 150K+ ctx). Problem: running multiple opencode agents (orchestrator, fixer, oracle) concurrently destroys each other's KV cache and prefix caching collapses. They ask whether a serial harness exists — supervisor delegates, sleeps, and wakes only after a subagent finishes and writes a full handoff — keeping one process active at all times to preserve prefix caching.
More from coding & agent
- Modal rebuilds sandbox platform to run 1M concurrent agent sandboxes in under a minute — charles_irl · 2026-09-14
- Dev hails opencode2 + DeepSeek V4.1 Flash as an insanely fast coding combo — Scobleizer · 2026-09-14
- Builder of 400+ production agents shares a decision tree for agent frameworks — MaryamMiradi · 2026-09-14
- Gemini 3.8 Flash with Antigravity draws elaborate diagrams right in the terminal — doodlestein · 2026-09-14
- $100/month coding agent beats buying a $10 app, quips AI researcher Yoav Goldberg — yoavgo · 2026-09-14
- SkySynth synthesizes formally verified systems: KV stores 2.3x faster than Redis — CShorten30 · 2026-09-14