kvdiff (MIT): a 63kB local proxy that byte-pinpoints MCP-induced prompt cache busts and auto-heals them
BringinItDirty · reddit · 2026-11-22
The author open-sourced kvdiff (MIT, 63 kB), a local proxy that diagnoses prompt-cache busts in AI coding agents — a silent cost multiplier for Codex, Claude Code and llama.cpp/vLLM users.
The problem: Anthropic, OpenAI (1,024-token prefix floor) and local KV-cache servers require a byte-for-byte identical prefix. If an MCP client reorders tools[], mutates schema key order, or injects a volatile timestamp into system prompt, cache hit rate collapses from 94% to 0%, forcing full prompt re-evaluation each turn. CLI transcripts don't expose the raw wire JSON, so the culprit is nearly impossible to find.
What kvdiff does:
- Sits between agent and API, computes the exact diverging UTF-8 byte between turns and maps it to a JSONPath (e.g. $.tools[9].name at byte 1,670)
- Classifies root causes: TOOLREORDER, TOOLMUTATED, VOLATILESYSTEM, PREFIXTOOSHORT, TTLEXPIRED, MODELCHANGED — with wasted-spend tracking in a local UI at 127.0.0.1:4318
- Zero-mutation passthrough by default (preserves OAuth/beta headers, streams SSE unbuffered)
- Opt-in --heal: deterministically sorts tools[] and schema keys, never touching messages/input/content/enum
Node >= 24, built-in node:sqlite, strictly binds to 127.0.0.1, zero telemetry. Try npx @drod915/kvdiff --demo (no API key needed).
More from coding & agent
- Antigravity team digests 500+ user feedback items; cross-workspace permission friction flagged — dotey · 2026-11-22
- LangChain's 'knowledge factory': better knowledge, not better models, is the missing bit — hwchase17 · 2026-11-22
- kvdiff (MIT): a 63kB local proxy that catches MCP tool-reordering cache busts and auto-heals them live — BringinItDirty · 2026-11-22
- Open-source quantified-self MCP server queries Apple Health, WHOOP and Oura locally in Claude Desktop — thecimal · 2026-11-22
- Alchemy Run's distilled SDK now beats first-party AWS SDK v3 on load time — samgoodwin89 · 2026-11-22
- Open-source ComfyUI nodes for Rumble2Flash, a faster and more accurate Qwen3-ASR-based transcriber — jmbbao · 2026-11-22