AWS tests four Bedrock settings that cut inference cost: cache TTL, Flex tier, effort, maxTokens

AWS ML Blog · rss · 2026-11-13

AWS ML Blog breaks down four request-level Amazon Bedrock choices that still move your bill after prompt/app tuning, with test data for each.

1. Prompt cache TTL

5-min writes cost 1.25x input rate, 1-hour writes 2x, cache reads 0.1x (Claude Sonnet/Opus 5)
1-hour cache pays off when prefix requests recur with >5-min gaps, 2-3+ times per hour
Checkpoints evaluate tools → system → messages; changing an earlier section invalidates later ones. Multi-tenant: prepend a tenant-specific SHA-256 hash to the prefix

2. Service tiers (Flex)

For latency-tolerant batch/eval workloads; Nova 2 Lite Flex is half the Standard rate in us-west-2
Unsupported models fail with ValidationException (Claude Sonnet 5 tested); verify via CloudWatch ResolvedServiceTier

3. Thinking effort

Claude Sonnet 5 / Opus 5 default to adaptive thinking at high effort; reasoning tokens bill as output tokens
Tool-calling agent tests: high effort used +169% output tokens vs low, max +300%+, with identical 24/24 correctness — lowering effort saves a lot

4. maxTokens

Includes request signatures, sample notebooks, and how to verify effects via response metadata, logs, and billing data.

More from coding & agent