20-turn agent loops make $20/mo SaaS seats unprofitable by turn 14 — caching + routing fixes it
TokenMarginDev · reddit · 2026-11-02
TokenMarginDev modeled cumulative token costs across 61 production LLMs for 20-turn agent loops (3,500 base context + 800 output tokens per turn):
- A single-turn chat bills 3,500 input tokens; a 20-turn agent loop bills 146,000 cumulative input tokens (14.5x). Small-context models like Phi-4 (16.4K) overflow by turn 5.
- Uncached, GPT-4o is 27% cheaper than Claude 3.7 Sonnet on a 3:1 workload. With 65% prompt-cache hits, Anthropic's 90% cache-read discount ($0.30/1M vs OpenAI's $1.25/1M) flips the math: $22.46 vs $22.53 per seat/month.
- 80/20 hybrid routing — 80% of routine turns to Gemini 2.5 Flash ($0.15/$0.60) or DeepSeek V3 ($0.24/$0.79), 20% of complex reasoning to Claude Sonnet 4 — cuts seat COGS from $22.50 to $4.60, restoring a 77% gross margin.
The author released an interactive cost calculator, a starter litellmconfig.yaml, and a zero-dependency TypeScript fallback router, and asks how others cap P95 power users: hard credit limits or silent model downgrades.
More from coding & agent
- Grok Bot template ecosystem grows: meetmybot aggregates community templates — RachelVT42 · 2026-11-02
- Trust is the #1 agent feature: accuracy, security, reliability beat everything else — cameronstow · 2026-11-02
- 9 of 10 AI agents secretly raised their own spending caps in safety test — shoumikgoswami · 2026-11-02
- Y Combinator-backed Jinko launches travel search and booking infrastructure for agents — ycombinator · 2026-11-02
- Grok Bot template roundup: Home Assistant watchdog, Quarto blog shipper, X unfollow tool — RachelVT42 · 2026-11-02
- Harnis: an open-source call-level seatbelt for local stdio MCP servers — L00pLegend · 2026-11-02