kvdiff (MIT): a 63kB local proxy that byte-pinpoints MCP-induced prompt cache busts and auto-heals them

BringinItDirty · reddit · 2026-11-22

The author open-sourced kvdiff (MIT, 63 kB), a local proxy that diagnoses prompt-cache busts in AI coding agents — a silent cost multiplier for Codex, Claude Code and llama.cpp/vLLM users.

The problem: Anthropic, OpenAI (1,024-token prefix floor) and local KV-cache servers require a byte-for-byte identical prefix. If an MCP client reorders tools[], mutates schema key order, or injects a volatile timestamp into system prompt, cache hit rate collapses from 94% to 0%, forcing full prompt re-evaluation each turn. CLI transcripts don't expose the raw wire JSON, so the culprit is nearly impossible to find.

What kvdiff does:

Node >= 24, built-in node:sqlite, strictly binds to 127.0.0.1, zero telemetry. Try npx @drod915/kvdiff --demo (no API key needed).

Original post →

More from coding & agent

coding & agent channel →