Why Your Agent Bill Exploded: The Quadratic Cost of Context Windows
Warm-Reaction-456 · reddit · 2026-08-11
A developer breaks down why a client's Agent API bill surged 9x: the quadratic cost of context windows. In an agent loop, every turn resends the entire history, meaning early tokens get billed repeatedly. Doubling the run length roughly quadruples the reading cost.
The Black Hole
Logs showed a typical run carried a 14k token base, with 11k just for tool schemas. A 38k token search result was reread 26 times, generating a massive bill. Reasoning is nearly free; re-reading is the real cost.
Optimizations
- Evict Results: Remove tool results from context once the step is done.
- Externalize Bulk: Write bulky content to a file, keeping only the path.
- Dynamic Tools: Shrink tool loadout to only what the task needs.
- Compaction: Compact long runs when summarizing is cheaper than rereading.
Post-optimization, the average run now bills 6 tokens for every unique token added, down from 18.
More from coding & agent
- How Do You Test AI Agents Before Production? Devs Discuss Hallucinations & Edge Cases — Saurabh4266 · 2026-08-11
- Developer Shares Workflow of Building an App Without Reading a Single Line of AI-Generated Code — tawnniee · 2026-08-11
- Browser Use CLI 3.0 Integrated into Hermes, Slashing Token Spend by 60% — Teknium · 2026-08-11
- Hermes Agent Launches Browser Automation Toolset with Multi-Backend Support — NousResearch · 2026-08-11
- Open Claw Agent Runs Overnight to Secure Fully Booked Health Club Spot — gregmushen · 2026-08-11
- Weights & Biases launches agent tracing that follows sessions, turns, and tool calls — wandb · 2026-08-11