Inference Costs Have Become Unsustainable
zeeg · x · 2026-07-14
The author wrote a long post discussing how their token consumption over the past week exceeded $10,000, pointing out that this level of inference cost is unsustainable.
The core of the article breaks down where these expenses come from and why "being able to afford running the model" doesn't equate to "being able to afford running a business." It focuses heavily on the cost structure on the inference side, the pressure brought by usage growth, and how to redesign products and calling methods to control expenses.
More from Infra
- Ubicloud publishes "Fixing Tool-Call Accuracy in vLLM" — petrusenko_max · 2026-10-26
- Meta approves nonvoting Class C shares as 2026 capex guidance hits $135B–$145B — thione · 2026-10-26
- Unsloth Ships MiniMax-M3.1-Flash Quants: 2-bit Retains 91.5% Accuracy, Runs in 150GB RAM — danielhanchen · 2026-10-26
- Engineer with thousands of fabricated devices pours cold water on photonic crystal modulator yields — jwt0625 · 2026-10-26
- Anthropic S-1 reveals 48 cents of compute cost per dollar of Q2 revenue — nmasc_ · 2026-10-26
- Persistent AI joins UR network to route private AI inference with encrypted, IP-hidden traffic — 0xSammy · 2026-10-26