4 semantic cache setups tested on 90k real support tweets — verifier wins 5-13% more hits
Reasonable_Royal_621 · reddit · 2026-11-04
The author replayed 90k real customer support tweets (AmazonHelp, Comcast) in time order, comparing Redis static threshold, GPTCache, vCache and CacheVerifier — same embedding, same cache, only the hit decision differs, judged by an LLM with manual spot checks.
- GPTCache's default threshold (cosine 0.6) hit 80% of requests but most replies were wrong for the new customer; Redis default (0.9) was safe at 1% error but only 4-5% hits.
- Fine-tuned CacheVerifier got 5-13% more hits than the best static threshold at the same error rate, staying closest to its 2% target as traffic shifted.
- Since support replies depend on conversation context, everything tops out around 5-9% hits.
Full writeup and scripts are open-sourced on GitHub.
More from Infra
- SurrealDB 4 launches as multi-model ACID database for the agentic era — Scobleizer · 2026-11-04
- Fire at Taiwan Panasonic PCB materials plant may tighten CCL supply further — zephyr_z9 · 2026-11-04
- Same Claude model served by Bedrock one day and Anthropic the next, dev finds — georgos_pappas · 2026-11-04
- Self-improving software is inevitable: closing the agent-observability loop — bibryam · 2026-11-04
- Gigabyte's AI TOP motherboard packs 2TB RAM and 9 GPUs for home AI labs — vincent_koc · 2026-11-04
- Building a 7-GPU 132GB VRAM local inference rig: consolidate on EPYC or split across two systems? — infectiousstupidity · 2026-11-04