A Tool for Debugging LLM Cache Misses
t4a8945 · reddit · 2026-07-18
The author built a debugging tool called cache-hunter for local-first LLM harnesses to catch exactly which changes in API calls trigger cache invalidation.
The Problem It Solves
When running models locally, prefill costs become glaringly obvious due to cache misses. Common culprits include:
- Message order changes
- Modified system prompts
- Altered tools
- Even shifts in reasoningeffort can trigger a miss
How to Use
- Start cache-hunter and point it to your real LLM endpoint
- Redirect your harness requests to cache-hunter's local port
- Click Start capture
- Run a normal session
The tool then displays the session in real-time, highlighting unstable fields in red.
Author's Observations
The author tested it on their own harness alongside OpenCode, Claude Code, Cline, Pi, Hermes, and Vibe. Many exposed instabilities in:
- System prompts
- Tools
- Sequencing
- Content
The author concludes that this kind of cache stability check should become a standard test for harnesses.
More from coding & agent
- Harness bug masked Gemini 4 Argon: rerun with Antigravity beats Astra, nears Opus 5.5 — vitaliychiley · 2026-10-27
- Design Is Becoming Game Dev: Build Your Own Tools With AI as the Executor — hewarsaber · 2026-10-27
- Dev turns DJI Mavic into local AI security drone built with Opus 5.5 — alexcovo_eth · 2026-10-27
- Microsoft Research: ForkPilot cuts long-horizon agent compute by up to 59% — EliasEskin · 2026-10-27
- Cursor launches Agent SDK: agents as code with Markdown prompts and TypeScript tools — soleio · 2026-10-27
- Vibecheck, the 'usable formal methods' hackathon, sells out two floors in SF — spikedoanz · 2026-10-27