Sarah Hooker: Test-time compute and agentic workflows are forcing AI infra to change
sarahookr · x · 2026-09-02
Why AI infrastructure is under pressure to change
Sarah Hooker (Cohere co-founder) argues the past decade's obsession was co-locating as much compute as possible, because:
- Pre-training large models is insanely cost-intensive
- Inter-GPU data transfer was the most unreliable bottleneck; runs must not fail
Two new forces are now stressing this architecture:
- Test-Time Compute — shifting demand to inference
- Agentic Workflows — task patterns that don't fit the co-located design
The infrastructure playbook built for pretraining is being rewritten.
More from Infra
- Auction-based LLM routing paper finds fixed token pricing wastes 10%-71% in costs — _onionesque · 2026-10-18
- Running Ling 3.1 Flash (560B MoE, 25B Active) Across 4 DGX Sparks via llama.cpp RPC — BerryBlush_Rogers691 · 2026-10-18
- SGLang Summit sponsors jump from 11 to 36 in one week — ying11231 · 2026-10-18
- Oracle CDS hits record 300bps, called first AI trade domino after Barclays sell downgrade — arthurcolle · 2026-10-18
- Agent Coding Workflow Would Cost ~$40k/Month on API Pricing, Dev Reveals — ChrisRackauckas · 2026-10-18
- PS5LM: llama.cpp port turns a jailbroken PS5 into a home LLM server (500+ GitHub stars) — wateriscoding · 2026-10-18