Redis LangCache claims 90% LLM cost cuts; KV, prefix, prompt and semantic caching explained
blaizedsouza · x · 2026-09-10
This shared article systematically explains the four cache layers in LLM apps — KV cache, prefix cache, prompt cache and semantic cache — and which repeated computation each one eliminates.
Core idea: production LLM apps rarely get entirely new questions. A customer-support bot may receive "Can I get a refund on the monthly plan?", "Is the monthly subscription refundable?" and "Can I cancel and get my money back?" — different wording, same underlying question and answer. Yet most apps treat every variant as a fresh request: reassemble the prompt, call the model, regenerate an answer that already exists.
Prefix caching only partially helps: shared system prompts can reuse computed KV states, but the request still hits the model — new tokens are processed and the full answer is still decoded. Semantic caching can return an existing answer directly on semantically similar hits, skipping inference entirely.
The linked product is Redis LangCache, positioned to save tokens on common questions, alongside Redis Iris (real-time agent context), Agent Memory, and Context Retriever components.
More from coding & agent
- AI agents don't know their code makes codebases harder to work with — _Stocko_ · 2026-09-10
- Hy4 preview tested: playable 3D survival game from a single prompt in WorkBuddy — mhdfaran · 2026-09-10
- Astra xhigh's token efficiency may stem from its subagent instructions — Al_Grigor · 2026-09-10
- Paper unveils preliminary Agent Swarm RL recipe; V4.1 teams hit 30% on ProgramBench — inductionheads · 2026-09-10
- AI-generated code maintenance costs up 300% in 18 months as firms 'let the agents cook' — bendee983 · 2026-09-10
- DeepSeek V4.1 Flash vs GPT-6 Astra: same coding task, wildly different code structures — kevinkern · 2026-09-10