A $1 "hi": Why Agentic AI sessions get expensive when cache goes cold
arena · x · 2026-08-19
During an agentic session, a quick "hi" after a break can cost a full dollar because you are paying to reload millions of context tokens after the cache goes cold.
Key points from the caching math analysis:
- Agentic AI involves significantly more back-and-forth than standard chat.
- You pay full price for the entire context window on a cache miss, even for a 1-token tool call.
- Cost grows quadratically even with caching, though hits reduce it to 10% of full price.
- Context compaction (e.g., Claude Code compacting to 200-300K tokens instead of 1M) helps mitigate costs.
Related event: Cold Agent Context Caches Turn a Simple 'Hi' Into a $1 Message(2 posts)→
More from Infra
- MCDMA Enables Direct RDMA Between NVIDIA Spark and Apple Silicon — StephanSturges · 2026-08-19
- Cursor's deep dive on Git at any scale: running Origin like a database — vmg · 2026-08-19
- Gemma 4 Powers Local Enterprise RFP Agent — victormustar · 2026-08-19
- Vector database Qdrant surpasses 250M downloads, recognized by Forrester — qdrant_engine · 2026-08-19
- Mojo programming language is now open source, combining Python ease with C++ performance — mgill25 · 2026-08-19
- Infra Behind Krea 2: Tensor Cores and Crash Philosophy — AI Engineer · 2026-08-19