Inference has no yardstick: 4.5-quadrillion token run rate, but no proof of what buyers get
AccBalanced · x · 2026-10-15
A CipherTalk deep-dive argues the booming AI inference industry lacks independent evidence of value:
- OpenRouter routes tokens at a 4.5-quadrillion annual run rate; Fireworks passed $1B; Baseten reportedly at a $26B valuation; CoreWeave reports $104B backlog; Nscale's IPO filing shows $103.4B contracted — only $2.6B active.
- Nscale spent more than it earned in H1 2026 ($189.6M cost vs $140.6M revenue); almost nobody publishes fleet utilization or inference gross margins.
- Pricing and performance are opaque: H100 hours list at $2.20 (DeepInfra) vs $8.00 (Fireworks); GPT-OSS-120B runs 36x faster on Cerebras than the slowest host, and the cheapest hosts score higher on accuracy.
- Existing benchmarks (MLPerf, Artificial Analysis, InferenceX) each measure a slice but can't tell buyers which system delivers the most work per dollar.
The authors propose a real yardstick needs: a metric matching how inference is bought, an independent benchmark, and contracts enforcing results.
Related event: AI Inference Market Booms in Opacity Amid Trillion-Token Demand(2 posts)→
More from Venture
- Gartner says 80% of neoclouds will die by 2030 — investor argues it's a supply problem, not demand — JOBhakdi · 2026-10-15
- Former Kleiner Perkins partner Bucky Moore strikes out to raise his own fund — nmasc_ · 2026-10-15
- Comparison pages are among the highest-ROI plays for winning AI search — jia_seed · 2026-10-15
- Investor: AI-generated cold emails and pitch decks get auto-deleted — dauber · 2026-10-15
- Why should interest rate expectations move AI stocks at all? — inductionheads · 2026-10-15
- Perplexity Computer Adds Polymarket Integration Covering 250,000+ Prediction Markets — SuhailKakar · 2026-10-15