Sarah Hooker: Test-time compute and agentic workflows are forcing AI infra to change
sarahookr · x · 2026-09-02
Why AI infrastructure is under pressure to change
Sarah Hooker (Cohere co-founder) argues the past decade's obsession was co-locating as much compute as possible, because:
- Pre-training large models is insanely cost-intensive
- Inter-GPU data transfer was the most unreliable bottleneck; runs must not fail
Two new forces are now stressing this architecture:
- Test-Time Compute — shifting demand to inference
- Agentic Workflows — task patterns that don't fit the co-located design
The infrastructure playbook built for pretraining is being rewritten.
More from Infra
- WSL nested virtualization is coming: team implements it after user's PoC PR — unixterminal · 2026-09-03
- Vercel Fluid Compute Now Runs 15M Builds Daily, Unifying Functions, Sandboxes and Builds on One System — soleio · 2026-09-03
- Baseten, NVIDIA Dynamo and SGLang to host SF meetup on RL post-training infrastructure — BanghuaZ · 2026-09-03
- Running Code OSS in a Cloud Run instance for a few bucks a month — rseroter · 2026-09-03
- Matthew Berman: data centers might get solved — Matthew Berman · 2026-09-03
- One startup's token usage jumped 720x in two months, from 1B per month to 1B per hour — MartinGTobias · 2026-09-03