A $1 "hi": Why Agentic AI sessions get expensive when cache goes cold
arena · x · 2026-08-19
During an agentic session, a quick "hi" after a break can cost a full dollar because you are paying to reload millions of context tokens after the cache goes cold.
Key points from the caching math analysis:
- Agentic AI involves significantly more back-and-forth than standard chat.
- You pay full price for the entire context window on a cache miss, even for a 1-token tool call.
- Cost grows quadratically even with caching, though hits reduce it to 10% of full price.
- Context compaction (e.g., Claude Code compacting to 200-300K tokens instead of 1M) helps mitigate costs.
Related event: Cold Agent Context Caches Turn a Simple 'Hi' Into a $1 Message(2 posts)→
More from Infra
- Prediction: Opus 5.5-level models on laptops by 2027, backed by Qwen 27B data — cephaloform · 2026-10-16
- Oracle down 12% in a week, nearly 50% off June high as OpenAI exposure spooks market — alvelda · 2026-10-16
- US data center moratoriums top 400 — an AI bull explains why they're actually good for AI — kevinsxu · 2026-10-16
- Microsoft Research unveils Magnetar, the fastest zkVM at 10-39x speedups and post-quantum security — jedisct1 · 2026-10-16
- Bonsai 2 2B hits 120 tok/s on iPhone 18 Pro, outpaces Qwen 3.5 2B at half the memory — cephaloform · 2026-10-16
- Samsung cuts Q4 phone output as Nittobo shifts E-glass to AI-chip T-glass — zephyr_z9 · 2026-10-16