Anthropic: skip RAG for knowledge bases under 200K tokens, cut costs up to 90% with prompt caching
CatAstro_Piyush · x · 2026-11-23
Anthropic's engineering blog lays out two ways to handle knowledge bases:
- Skip RAG entirely: if your knowledge base fits within 200K tokens (500 pages), just include the full text in the prompt. Combined with Prompt Caching, this reportedly improves latency by over 2x and cuts costs by up to 90%.
- Contextual Retrieval for larger bases: using Contextual Embeddings plus Contextual BM25, failed retrievals drop 49%, and 67% when combined with reranking.
The idea leverages long context windows to cache background knowledge as a prefix, sidestepping the context loss that plagues traditional RAG encoding. A cookbook is available for direct deployment.
More from coding & agent
- Someone steered Claude Code at proving the Riemann hypothesis — prasanna_says · 2026-11-23
- Microsoft's Spec Kit championed for spec-driven development with AI coding agents — luisdans · 2026-11-23
- AI Agent Designs Complete RISC-V CPU Core From Scratch in 12 Hours — ChuckDBrooks · 2026-11-23
- Agent infrastructure revenue boom: E2B hits $120M, Daytona $100M — clemnt · 2026-11-23
- Free Claude plugin turns your MCP-connected video models into an MV factory — AIandDesign · 2026-11-23
- Andrew Ng's AI Dev NYC: agent memory, coding agents, 35+ speakers, Nov 30–Dec 1 — DeepLearningAI · 2026-11-23