Bedrock adds explicit prompt caching for open-weight models, cutting input costs
AWS ML Blog · rss · 2026-11-16
AWS ML Blog explains using prompt caching with open-weight models on Amazon Bedrock to cut input costs and latency for repeated long contexts.
Two modes
- Implicit caching: automatic best-effort prefix reuse, ideal for conversations and agent loops; put static content first, dynamic last
- Explicit caching: mark reusable prefixes with cache breakpoints via promptcacheoptions on the Chat Completions API; multiple breakpoints support layered caching, avoiding higher cache-write fees for content never reused
Billing: cache reads bill below standard input; cache writes may bill higher; TTL resets on each hit. The post walks through caching a 4,000-token product manual with the OpenAI Python SDK plus short-term Bedrock API keys, applying implicit caching to an OpenCode coding agent session, and reading cache usage fields from responses.
More from coding & agent
- LangChain founder: agents are shifting from executing tasks to owning goals — hwchase17 · 2026-11-17
- Wajo's Fo gives every person a digital twin with its own phone number and email — khant_dev · 2026-11-17
- 58M-line Chromium codebase fully read and analyzed in under 50 seconds — sull · 2026-11-17
- Singularity-Bench: Opus 5.5 with Claude Code beats human baselines on building AI systems — armand_ruiz · 2026-11-17
- AI platform security checklist: default gateway keys, open inference endpoints, non-expiring tokens — mclynd · 2026-11-17
- Every staffer clones her editors from Google Docs edit history in a word processor built on Fable 5.2 — danshipper · 2026-11-17