Agentic Loops Shift AI Infra Focus From FLOPs to the von Neumann Bottleneck
sarahookr · x · 2026-09-03
Sarah Hooker argues that babysitting multi-step agentic loops requires constantly offloading intermediate states, so the future of AI infrastructure is not just maximizing FLOPs but innovating on the von Neumann bottleneck—how fast data can move back and forth. Implications:
- Datacenter designs will change.
- Hardware in space is probably not a good idea.
- It partly explains why consumer laptops have gone up in price—memory markets are extremely expensive right now.
More from Infra
- WSL nested virtualization is coming: team implements it after user's PoC PR — unixterminal · 2026-09-03
- Vercel Fluid Compute Now Runs 15M Builds Daily, Unifying Functions, Sandboxes and Builds on One System — soleio · 2026-09-03
- Baseten, NVIDIA Dynamo and SGLang to host SF meetup on RL post-training infrastructure — BanghuaZ · 2026-09-03
- Running Code OSS in a Cloud Run instance for a few bucks a month — rseroter · 2026-09-03
- Matthew Berman: data centers might get solved — Matthew Berman · 2026-09-03
- One startup's token usage jumped 720x in two months, from 1B per month to 1B per hour — MartinGTobias · 2026-09-03