Factoring in Retries: Are Cheaper AI Models Actually Cheaper?
ExtremeAdmirable4097 · reddit · 2026-08-06
A developer compared GPT-5.6 Luna and DeepSeek V4 Flash for generating a web game. They found that while DeepSeek's nominal per-call cost was a fifth of GPT's, factoring in the cost of failed retries significantly increased the actual spend to reach a successful result.
The post points out that real AI systems must pay for failed calls, retries, and human reviews. Thus, looking solely at inference price is insufficient. The author calls for a more robust benchmarking standard that reports both 'cost per attempt' and 'cumulative cost to first verified success', while separating visual success from functional correctness.
More from coding & agent
- Exploring Recursion Depth and Persistent Context in AI Agents — inductionheads · 2026-08-06
- The 80/20/0 Rule for AI Coding: Balancing Speed and Security — alex_verem · 2026-08-06
- Stop Saying 'Make It Better': Build Scoring Loops for AI Agents — Roger_M_Taylor · 2026-08-06
- Microsoft's PlugMem Slashes Agent Context by 100x via Knowledge Graphs — Roger_M_Taylor · 2026-08-06
- Run Multiple Models on One GPU: SIE Cuts Self-Hosting Costs 75% — Roger_M_Taylor · 2026-08-06
- OpenTag: Open-Source AI Agent for Quick Slack Deployment — Roger_M_Taylor · 2026-08-06