Sara Hooker: The Future of AI Infra Is Beating the von Neumann Bottleneck, Not FLOPs
sarahookr · x · 2026-09-03
Sara Hooker argues that babysitting multi-step agentic loops requires constantly offloading intermediate states, fundamentally changing infrastructure demands.
Her core points:
- Traditional inference is like getting dough to the oven — it's about maximizing FLOPs;
- Agents operate in multi-step loops — execute, check success, adjust — like a factory needing to move cookies in and out of the oven repeatedly;
- The future of infra is therefore not just maximizing FLOPs but innovating on the von Neumann bottleneck — how fast you can move data back and forth — historically the hardest problem in hardware architecture.
More from AGI Musings
- William Tunstall-Pedoe: The 'Trust Ceiling' — Trillions In Value Stuck Behind Unreliable AI — williamtp · 2026-09-03
- Nebula-winning novelist R.F. Kuang on writing: the feel of writing predicts nothing — david_perell · 2026-09-03
- BART Costs $48.76 Per Trip with $43.58 Subsidized — Making the Case for Self-Driving Transit — garrytan · 2026-09-03
- Only 15% of Bank AI Use Cases Reach Production, 60% Stuck in the 'Frozen Middle' — mikeflache · 2026-09-03
- Pedro Domingos: We need Goodhart-proof measures for AI evaluation — pmddomingos · 2026-09-03
- New hypothesis paper argues consciousness and cognition are separate evolutionary lineages, with six falsifiable predictions — rjhaier · 2026-09-03