Agents pull AI compute back to the CPU: AWS, Google, Microsoft, NVIDIA all build on Arm
kimmonismus · x · 2026-11-20
The thread argues agent workloads are pulling AI hardware back to the CPU: the model thinks on the GPU, but spinning up sandboxes, running code, calling tools, and deciding next steps all run on the CPU — once per answer for chatbots, looping endlessly for agents. AMD says chatbot-era servers ran 1 CPU per 4–8 GPUs; with agents the ratio moves toward 1:1.
Arm's agent-workload benchmarks of Cobalt 200 vs AMD EPYC Turin: up to 46% more SWE-bench tasks/min, 56% higher Milvus vector search, 57% higher fastText. All four rivals — AWS (Graviton), Google (Axion), Microsoft (Cobalt), NVIDIA (Grace/Vera) — build on Arm; Google alone ported 30,000+ apps, and Arm now sells its own AGI CPU co-developed with Meta.
More from coding & agent
- Drex Router cuts small-game build cost to $1.26 vs $13.44 on Claude Opus alone — testingcatalog · 2026-11-20
- User teaches AI companion to ignore "drop it" said to his dog, then fixes the over-pause — manosaie · 2026-11-20
- NVIDIA Dynamo cuts RL rollout bottlenecks with KV-aware routing and delta weight updates — NVIDIAAI · 2026-11-20
- LangChain Academy Answers: Can a Better Model Fix a Broken Agent? — LangChain · 2026-11-20
- Coding agent index shows max effort costs 6x for fewer points; Gemini 4 Neon wins on value — Arindam_1729 · 2026-11-20
- Drex Router: a small model that picks the right LLM for every agent step — omarsar0 · 2026-11-20