ArisGlobal cut Bedrock inference cost bloat by caching shared prompt prefixes first

AWS ML Blog · rss · 2026-11-17

ArisGlobal's NavaX extracts 130 fields from adverse-event PDFs on Amazon Bedrock. As per-tenant rules grew the shared prompt prefix from 72K to 180K tokens (+150% in 6 months), per-case inference costs rose 200%+. The fix: reverse the request order — send the shared prefix first with an explicit cache checkpoint, and put the per-case document after — so the prefix is reused across requests and tenants, enabling sublinear cost growth. Key details: prefix-based caching, up to 4 checkpoints per Claude request, 1024-token minimum prefix, 1-hour TTL support. The system has processed 1M+ real transactions in regulated pharma environments.

Original post →

More from coding & agent

coding & agent channel →