Sam Altman: local inference is terrible resource allocation—your DRAM sits idle 95% of the time
FinanceYF5 · x · 2026-10-30
Sam Altman argues that in a resource-constrained environment, local inference is poor global resource allocation: putting large amounts of scarce DRAM in everyone's computer while it's used only 5% of the time is highly inefficient. When supply is tight, he says, what's needed is keeping cloud resources at maximum utilization—adding that many people's intuitions run exactly the opposite way.
More from Infra
- Nvidia weighs cosigning loan for Nscale's West Virginia AI data center — TechCrunch AI · 2026-10-31
- Northflank dunked on Railway's breach, then got hacked the same way 9 days later — aniketmaurya · 2026-10-31
- Dev says Neon is already chasing millisecond infra branching, wants it built into alchemy — samgoodwin89 · 2026-10-31
- Big AI labs all push compute efficiency: Trainium power savings, small models, cache compression — WillfullyWorkable · 2026-10-31
- Running 12 agents on one RTX 3090 lifts total throughput from 30 to 110 tok/s — Mchanger · 2026-10-31
- Analyst turns ultra-bull on Avicena after 6 hours of technical diligence — BenBajarin · 2026-10-31