AWS on agent prompt caching: explicit + implicit patterns to slash token costs
AWS ML Blog · rss · 2026-10-27
AWS ML Blog walks through prompt caching for agents using OpenAI models on Amazon Bedrock.
- Problem: agents resend their entire context (tools, instructions, full history) every turn — by turn 20 the model re-reads turns 1–19, and you pay for all of it
- Mechanics: caching saves the KV state built during prefill; identical prefixes are reused at a fraction of input price
- Explicit mode: mark a static tools+instructions prefix (≥1024 tokens) with promptcachebreakpoint and share it via promptcachekey; a 4,000-token prefix wrote 4,120 tokens once, then only reads
- Implicit mode: Bedrock auto-selects the cache boundary as the conversation grows
- Recommendation: combine both for a shared starting point plus growing history
- Gotchas: prefixes must match byte-for-byte, MCP tool serialization must stay consistent, and variable content (dates, user names) goes after the breakpoint
More from coding & agent
- Kolab Benchmark Uses Pro Screen Recordings to Show AI Agents Overestimate Collaboration Quality — minnesotanlp · 2026-10-27
- Dev argues Claude burns far more tokens than GPT models for the same coding work — haider1 · 2026-10-27
- Stop building agents that can't change models: stable contracts, external state, real evals — rseroter · 2026-10-27
- Cheap models can cost more in agent harnesses than frontier ones, researcher warns — vasuman · 2026-10-27
- Kanzo: A 5k-Line Rust Kanban Binary With MCP for Claude Code and Codex — allmudi · 2026-10-27
- 'A human approves everything' is a trap: approval fatigue and how to actually design safe agents — Efistoffeles · 2026-10-27