Evaluating AI Costs Requires Looking at Task Completion Rates
luckokkkk · reddit · 2026-07-15
The core point of this post is that AI costs cannot be calculated solely by token price; task completion rates and the total context consumed to actually complete a task must be factored in.
It cites Databricks' measurements: in the Claude Code scenario, an average of 742,000 context tokens are consumed per task, while Pi on Opus consumes about 236,999. The author uses this to illustrate that seemingly cheaper models or solutions may not actually be more cost-effective for the real cost of "completing the same task."
This type of comparison leans closer to "task-level cost" rather than "token-level cost," which is crucial for evaluating the economics of coding assistants, agentic workflows, and inference services.
Related event: True AI Cost Depends on Task Completion Rates(2 posts)→
More from Infra
- Michael Dell: Musk is "the most demanding customer there is" — Colossus servers built in ~6 weeks — elonmusk · 2026-10-27
- AI chip startup Tenstorrent raises $1.5B at $6.2B valuation, more than doubling — Beth_Kindig · 2026-10-27
- Ex-MediaTek expert: Google TPU external revenue $8B this year, $104B by 2028 — RihardJarc · 2026-10-27
- Anthropic discloses up to $120.5B SpaceX compute deal, ~$3.1B monthly at full capacity — XFreeze · 2026-10-27
- Satire: swapping Raft for an LLM-style decision model tanks write throughput 99.98% — irombie · 2026-10-27
- Cadence joins the domain-specific AI model camp for chip design — karlfreund · 2026-10-27