AWS and Accenture cut Bedrock token spend 40–85% with six optimization levers, from prompt caching to distillation
AWS ML Blog · rss · 2026-11-20
An AWS ML Blog post with Accenture details a sequenced approach to AI economics on Amazon Bedrock, where token spend often grows faster than per-token prices fall — the Linux Foundation even launched a Tokenomics Foundation for AI cost economics.
Attribution first: cost allocation tags via application inference profiles, with CUR 2.0, CloudWatch, and invocation logging as telemetry; Accenture's Cost per Business Transaction (CpBT) metric rolls up all invocations, retries, tool calls and agent handoffs into a fully loaded cost per business output.
Six levers in three phases:
- Phase 1 (config only): prompt caching (up to 90% off cached input tokens) and batch inference (50% off on-demand)
- Phase 2 (architecture): intelligent prompt routing (up to 30% savings without accuracy loss), RAG optimization (hierarchical chunking, metadata filtering, reranking), plus Accenture's context scoping and context compression patterns for multi-agent chains
- Phase 3 (infrastructure/training): semantic caching — on 63,796 real chatbot queries a 0.90 similarity threshold gave a 74.5% hit rate, 92.3% accuracy, 72.5% cost savings; model distillation — up to 500% faster and 75% cheaper with <2% accuracy loss
In production: applied across telecom, insurance, mining and large-enterprise engagements, measured token spend fell 40–85% where savings were measured.
More from coding & agent
- The AI-era PRD template: prototypes show what, the PRD shows why and how to measure — aakashgupta · 2026-11-20
- Agent checks 40 startup names in one go for 3 cents via iMessage — socialwithaayan · 2026-11-20
- Amazon paper: stale agent skills can drop performance 10pp and raise cost 40% — rohanpaul_ai · 2026-11-20
- Amazon paper: stale agent skills cut pass rate 11.4 points, selective maintenance fixes it — rohanpaul_ai · 2026-11-20
- Specula renamed to Specular over unintended meaning, v1.5.0 released — tianyin_xu · 2026-11-20
- OpenAI ships Codex plugin for ModRetro Chromatic handheld to build games — OpenAIDevs · 2026-11-20