Prefixwise names why your prompt cache misses — 19 causes, recovers 99.9% of savings
No-Individual159 · reddit · 2026-11-09
Developer released Prefixwise (MIT, pip install prefixwise), a tool that sits between your agent and the model (or inside vLLM) to explain every prompt-cache miss in plain English, price it, auto-fix common causes, and block cache regressions in CI. Nothing leaves your machine — only hashes and counts.
- Identify: 19 miss causes (reordered tools, timestamps in system prompt, edited history, compaction rewrites, misplaced cache breakpoints, TTL expiry, routing misses, evictions…); agrees with Anthropic's own diagnostics 32/32
- Fix: --fix canonicaltools repairs in flight — measured 99.9% of predicted savings on Claude, 97.5% on vLLM, task success unchanged
- Prevent: prefixwise check -- pytest or a GitHub Action fails CI when a change breaks the cache
- Integration: wraps claude/codex/any agent, local proxy, vLLM middleware, Python SDK, LiteLLM callback; reads KV events from vLLM/SGLang/LMCache to confirm evictions and routing misses
- Live finding this week: reasoning-model chat templates that drop thinking from history re-prefill the previous answer every turn — Qwen3-4B wastes 25-36% of prompt tokens, a DeepSeek-R1 distill 57%, while Qwen2.5/Llama 3.2 keep history: 0%
Overhead is 3ms per request; every README number links to its run record.
More from coding & agent
- Dev builds full-screen voice face for Mac that drives Codex coding agents hands-free — Ok-Suspect-9855 · 2026-11-09
- DeepSeek agent harness wired to a Grok x_search plugin pulls live X posts, demo shows — teortaxesTex · 2026-11-09
- Agent passed its own tests while wrong: lessons from a 2-bit local model building a Unity game — Stunning-Sherbet1853 · 2026-11-09
- 12 GitHub repos poisoned with CLAUDE.md prompt injection tricking AI coding agents into running malware — cyb3rops · 2026-11-09
- How to roll out AGENTS.md, 50 skills and guardrail hooks to 500+ devs — KrishnaKA2810 · 2026-11-09
- Multi-subagent migration passed all tests, yet UTC dates billed a day early — comptedemon · 2026-11-09