Sam Altman: local inference wastes idle DRAM, datacenters win on per-token cost
rohanpaul_ai · x · 2026-10-30
Sam Altman argued that from a global allocation perspective, having the world's limited DRAM supply sit idle in personal computers 95% of the time is "quite terrible." Per token, datacenters win easily—a batch-size-1 local model is wasteful by any utilization metric. His one concession: local models are the only part of the stack nobody can deprecate on you.
More from Infra
- Nvidia weighs cosigning loan for Nscale's West Virginia AI data center — TechCrunch AI · 2026-10-31
- Northflank dunked on Railway's breach, then got hacked the same way 9 days later — aniketmaurya · 2026-10-31
- Dev says Neon is already chasing millisecond infra branching, wants it built into alchemy — samgoodwin89 · 2026-10-31
- Big AI labs all push compute efficiency: Trainium power savings, small models, cache compression — WillfullyWorkable · 2026-10-31
- Running 12 agents on one RTX 3090 lifts total throughput from 30 to 110 tok/s — Mchanger · 2026-10-31
- Analyst turns ultra-bull on Avicena after 6 hours of technical diligence — BenBajarin · 2026-10-31