ArisGlobal cut Bedrock inference cost bloat by caching shared prompt prefixes first
AWS ML Blog · rss · 2026-11-17
ArisGlobal's NavaX extracts 130 fields from adverse-event PDFs on Amazon Bedrock. As per-tenant rules grew the shared prompt prefix from 72K to 180K tokens (+150% in 6 months), per-case inference costs rose 200%+. The fix: reverse the request order — send the shared prefix first with an explicit cache checkpoint, and put the per-case document after — so the prefix is reused across requests and tenants, enabling sublinear cost growth. Key details: prefix-based caching, up to 4 checkpoints per Claude request, 1024-token minimum prefix, 1-hour TTL support. The system has processed 1M+ real transactions in regulated pharma environments.
More from coding & agent
- LangChain explains Agent vs. Deep Agent vs. Managed Deep Agent — LangChain · 2026-11-18
- Open-Source Robot Lamp Preps Skill Store: Install Physical Skills Like Apps — dee_hw · 2026-11-17
- Turning a Support Inbox Into an MCP Server So Claude Code Handles It Directly — Crazy-Mountain6125 · 2026-11-17
- Humans& Launches 744B-Param Collaborator Model Ampersand on Vercel AI Gateway — ericzelikman · 2026-11-17
- Programming tools are shifting from file-centric IDEs to spec- and voice-driven agent workflows — bendee983 · 2026-11-17
- Founder Launches an MCP Server for SMS and OTP Codes, Seeking Tool-Design Feedback — Comfortable_Flan_752 · 2026-11-17