AWS cuts agent token costs 71% and latency 80% on 23MB JSON workloads with three strategies
AWS ML Blog · rss · 2026-11-18
AWS ML Blog details how it built a production agent for airline ops platform Flightscape (40+ tools, 20MB+ JSON inputs) that originally exceeded 15s latency per call. Using three composable strategies, token consumption dropped up to 71% and average latency up to 80% on a representative 23MB payload.
Stack: LangGraph on Amazon Bedrock AgentCore with memory persistence; Bedrock Guardrails filters responses.
The three strategies:
- Lightweight session state, heavy artifacts offloaded: request/response/validation JSONs go to S3; AgentCore Memory stores only reference keys, hydrated on demand
- Large tool results stored in S3, model gets a compact summary + handle with paging via a get-result tool
- Dehydrate at end of turn: heavy payloads stripped from graph state, only light fields and references persisted
Includes TypedDict definitions for PersistedState/GraphState, code for dehydratestate, and a companion GitHub repo.
More from coding & agent
- Code Mode turns 1: every major agent harness has adopted it — and kept the name — ritakozlov · 2026-11-18
- GitHub and Microsoft's MDASH pilot turns thousands of AI vuln findings into 99 verified fixes — mustafasuleyman · 2026-11-18
- Simon Willison open-sources three external sandboxes for safer coding agents — Simon Willison · 2026-11-18
- OpenAI updates API dashboard with per-model usage, rate-limit and error tracking — OpenAIDevs · 2026-11-18
- 'Proof-Carrying Calls' Idea: Make Agents Certify They Don't Break Your Invariants — tianyin_xu · 2026-11-18
- Devin Platform Adds a 'Poke' Bot Template: An Engineering Agent with Personality — marvinvonhagen · 2026-11-18