Sam Altman: local inference is a 'terrible allocation' of scarce compute resources
GabGarrett · x · 2026-10-30
Sam Altman pushed back on local inference enthusiasm: many assume open-source models mean no more data centers, but leaving DRAM idle in personal machines used 5% of the time is, from a global allocation view, quite terrible in a supply-constrained environment. Quoter xlr8harder wonders whether lab behavior may have shaped these user preferences.
More from Infra
- Nvidia weighs cosigning loan for Nscale's West Virginia AI data center — TechCrunch AI · 2026-10-31
- Northflank dunked on Railway's breach, then got hacked the same way 9 days later — aniketmaurya · 2026-10-31
- Dev says Neon is already chasing millisecond infra branching, wants it built into alchemy — samgoodwin89 · 2026-10-31
- Big AI labs all push compute efficiency: Trainium power savings, small models, cache compression — WillfullyWorkable · 2026-10-31
- Running 12 agents on one RTX 3090 lifts total throughput from 30 to 110 tok/s — Mchanger · 2026-10-31
- Analyst turns ultra-bull on Avicena after 6 hours of technical diligence — BenBajarin · 2026-10-31