Prefixwise: intentionally broke prompt cache 1,772 times to validate root-cause tool
No-Individual159 · reddit · 2026-11-13
The author releases Prefixwise (MIT, open source): LLM servers (vLLM, SGLang, Anthropic, OpenAI) tell you how many tokens missed the prefix cache but not why; Prefixwise pinpoints which change (tool reordering, timestamps in the system prompt, rewording, edited messages) broke the cache, what it cost, and how to fix it.
Validation: 1,772 deliberately injected changes replayed through real engines (Qwen3-4B on a Colab L4):
- vLLM 0.33: 441/441 correct causes on generated traffic, 425/441 on real Claude Code sessions; 0 wrong causes, 0 false alarms (900 negative checks each)
- SGLang 0.5.23: 456/456 and 424/434 correct; 0 false alarms
- The 26 misses were edits to the newest tool output — the uncached tail, so no cost.
Half the traffic comes from 150 real Claude Code sessions (nvidia/Open-SWE-Traces). Usage: pip install prefixwise; stores only hashes and token counts. Repo: github.com/tejachavali/prefixwise
More from coding & agent
- With AI help, a team finally migrated all their infra to K8s after ragequitting years ago — DanielLockyer · 2026-11-13
- Agents trust other agents by default — one user's skeptical-agent fix for a real security hole — andrey_kurenkov · 2026-11-13
- A dead-simple canary test for drifting agent sessions: make it address you by name — brandon_galang · 2026-11-13
- Firecrawl MCP v3 launches with 3,200+ tools, 60x rate limits and parallel subtasks — gaganghotra_ · 2026-11-13
- Software abstractions are dead: agents now rewrite whole stacks per workload — DimitrisPapail · 2026-11-13
- Voice agent replaces onboarding questionnaires: rethinking SaaS for agentic platforms — strickvl · 2026-11-13