Enterprise AI Compute Purchasing Outpaces Cost Visibility
VentureBeat AI · rss · 2026-07-17
A VentureBeat survey reveals that while enterprise AI infrastructure spending is growing rapidly, many organizations lack clear control over costs and utilization, creating what the author calls a compute gap.
Key Findings
- Only 21% of surveyed enterprises are running AI at scale in production
- 83% report GPU utilization at 50% or below
- Only 44% can strictly track AI compute costs
- 64% plan to switch or add infrastructure providers in the next 12 months, with 38% planning to act within the next quarter
Current and Future Infrastructure Preferences
- The most commonly used currently are traditional clouds and model APIs: Google Cloud, Azure, AWS, Oracle, and Gemini / OpenAI / Anthropic
- Specialized AI clouds (like CoreWeave, Lambda, Crusoe, Nebius) currently hold a small share, but 45% of enterprises plan to evaluate these options in the future
- Next are non-NVIDIA accelerators (Trainium, TPU, AMD Instinct, Gaudi, ASIC) and Blackwell / next-gen GPUs
Selection Criteria
Enterprises prioritize:
- Integration with existing tech stacks (41%)
- Total Cost of Ownership (TCO) (35%)
Meanwhile, only 8% consider "cost per million tokens" a deciding factor. The article also notes that as inference scales up, the next bottleneck may shift from GPU compute to memory bandwidth, though most enterprises have yet to factor this into their planning.
More from Infra
- Nebius says SlimSpec speeds speculative decoding 8–9% without shrinking the vocabulary — Arindam_1729 · 2026-07-21
- NVIDIA brings its Cosmos 3 Edge world model to Jetson for on-device robot control — liu_mingyu · 2026-07-21
- A silicon photonic reservoir chip compensates fiber distortion in real time at 28 Gbps — bravo_abad · 2026-07-21
- Chamath says open-sourcing Grok would push AI margins from models to infra and apps — Dan_Jeffries1 · 2026-07-21
- EU AI competitiveness is under pressure as firms double down on chips, ethics, and talent — nordicinst · 2026-07-21
- AI bottlenecks are shifting to memory, optics, yield control and power — thedealdirector · 2026-07-21