Sara Hooker: The Future of AI Infra Is Beating the von Neumann Bottleneck, Not FLOPs
sarahookr · x · 2026-09-03
Sara Hooker argues that babysitting multi-step agentic loops requires constantly offloading intermediate states, fundamentally changing infrastructure demands.
Her core points:
- Traditional inference is like getting dough to the oven — it's about maximizing FLOPs;
- Agents operate in multi-step loops — execute, check success, adjust — like a factory needing to move cookies in and out of the oven repeatedly;
- The future of infra is therefore not just maximizing FLOPs but innovating on the von Neumann bottleneck — how fast you can move data back and forth — historically the hardest problem in hardware architecture.
More from AGI Musings
- When AI can clone your product by sight, paid software's moat is service and trust — sujingshen · 2026-10-18
- Anthropic borrows lawyers' ethics for Claude agents that closed 186 deals with no human approval — rohanpaul_ai · 2026-10-18
- Tim Sweeney: AI will unlock a new golden age of games too costly to build before — TinfoilTricorn · 2026-10-18
- Researcher argues we may solve AI-human alignment before humans align with each other — 3scorciav · 2026-10-18
- EigenGender: private-industry coup isn't the default, but worth guarding against — EigenGender · 2026-10-18
- Writing instructor's meltdown: "LLMs are literally just spicy autocomplete" — Decent-Decision-9028 · 2026-10-18