A Tool for Debugging LLM Cache Misses
t4a8945 · reddit · 2026-07-18
The author built a debugging tool called cache-hunter for local-first LLM harnesses to catch exactly which changes in API calls trigger cache invalidation.
The Problem It Solves
When running models locally, prefill costs become glaringly obvious due to cache misses. Common culprits include:
- Message order changes
- Modified system prompts
- Altered tools
- Even shifts in reasoningeffort can trigger a miss
How to Use
- Start cache-hunter and point it to your real LLM endpoint
- Redirect your harness requests to cache-hunter's local port
- Click Start capture
- Run a normal session
The tool then displays the session in real-time, highlighting unstable fields in red.
Author's Observations
The author tested it on their own harness alongside OpenCode, Claude Code, Cline, Pi, Hermes, and Vibe. Many exposed instabilities in:
- System prompts
- Tools
- Sequencing
- Content
The author concludes that this kind of cache stability check should become a standard test for harnesses.
More from coding & agent
- Soft Clamp cuts tool-call overuse in multi-teacher distillation, from 13.7% to 9.0% — antgroup · 2026-07-21
- Agent harness memory loss and compaction are still a major usability problem — adityaag · 2026-07-21
- SpecJudge runs locally on Ollama to pick the right-sized AI model for your project — jokiruiz · 2026-07-21
- A developer maps out six design rules for CLIs that humans and AI agents can both use — yujiezha · 2026-07-21
- A coding-agent skill that forces ADHD-friendly, answer-first output — ayghri · 2026-07-21
- A set of agent skills for CAD, robotics, and hardware design — earthtojake · 2026-07-21