A Tool for Debugging LLM Cache Misses
t4a8945 · reddit · 2026-07-18
The author built a debugging tool called cache-hunter for local-first LLM harnesses to catch exactly which changes in API calls trigger cache invalidation.
The Problem It Solves
When running models locally, prefill costs become glaringly obvious due to cache misses. Common culprits include:
- Message order changes
- Modified system prompts
- Altered tools
- Even shifts in reasoningeffort can trigger a miss
How to Use
- Start cache-hunter and point it to your real LLM endpoint
- Redirect your harness requests to cache-hunter's local port
- Click Start capture
- Run a normal session
The tool then displays the session in real-time, highlighting unstable fields in red.
Author's Observations
The author tested it on their own harness alongside OpenCode, Claude Code, Cline, Pi, Hermes, and Vibe. Many exposed instabilities in:
- System prompts
- Tools
- Sequencing
- Content
The author concludes that this kind of cache stability check should become a standard test for harnesses.
More from coding & agent
- Codex tip: use Sol with Astra and Luna sub-agents to save usage — pvncher · 2026-09-11
- agents-best-practices: a provider-neutral Agent Skill for designing and auditing agentic harnesses — tom_doerr · 2026-09-11
- Cognition's SWE-2 uses a KKT duality argument in RL to shift the effort Pareto curve — YouJiacheng · 2026-09-11
- First-ever Three.js Conference lands in Paris, with a panel on AI-shortened design workflows — OdinLovis · 2026-09-11
- Data engineering, not agent frameworks, is the real bottleneck for enterprise AI agents — dhruv2038 · 2026-09-11
- RTK Terminal Compression Cuts Tokens but Leaves Your AI Coding Bill Unchanged — Bartaseth · 2026-09-11