Inference providers hit $1M/mo fast: the arbitrage economics of renting GPUs to serve open-source tokens
deedydas · x · 2026-10-25
- Deedy notes a barrage of inference providers reaching $1M/mo revenue extremely quickly, with low and falling barriers to entry.
- Economics: even renting on-demand B300s at $8/GPU-hr, open-source models (GLM 5.3, Kimi K3, DeepSeek 4.1) can yield >$10-20/GPU-hr at reasonable throughput (100 tok/s); DeepSeek's DeepInfer shows $38/GPU-hr off-peak. The attached chart maps gross margin vs utilization per model/chip combo: GLM-5.3 on Vera Rubin hits 50% margin at 20% utilization, while GB300 combos need far more.
- Gateways like OpenRouter and Vercel aggregate demand; some even resell Chinese model tokens back to Chinese labs. But Chinese labs increasingly demand rev share at scale (especially for pre-release day-1 access), raising breakeven thresholds.
- As a commodity service, this should push compute prices up and token prices down. Neoclouds (Crusoe, CoreWeave, Nebius, GMI) and well-capitalized players (Modal, Fireworks, Baseten) should win, but the pie is growing so fast that upstarts like inference.net, DeepInfra, Wafer are crushing it too.
- Risks: competitiveness of open-source models, creator rent-seeking, changing compute shapes, chip economics, and who gets compute in a crunch.
More from Venture
- Brex data: agent compute rentals surge, Baseten tops fall 2026 fastest-growing vendors — FinanceYF5 · 2026-10-25
- Greg Isenberg lists 39 AI startup ideas: redo proven business models for the agent era — FinanceYF5 · 2026-10-25
- Silicon Valley notes: insiders already flipping Anthropic stakes while outsiders debate the bubble — dotey · 2026-10-25
- VC: Startups aren't supposed to make money — profits should fund growth — AccBalanced · 2026-10-25
- Robot IPO window closes: China's embodied AI firms stall after Unitree listing — 量子位 · 2026-10-25
- Musk says X's ultimate vision is to become 'maybe half of the global financial system' — XFreeze · 2026-10-25