Don't Run Sequential Workloads on LLMs — Have Them Write Cheap CPU Code Instead

cis_female · x · 2026-10-30

A discussion on LLM compute economics: sequential, branchy, long workloads run poorly on LLM inference — even with high MFU it's inefficient; CPUs are the right tool.
The counterpoint adds a neat twist: the correct way to use an LLM here is not to run the workload on it, but to have the LLM write code that executes cheaply on a CPU — the model generates the program, the CPU runs it.
A simple but memorable division of labor: models are good at writing code, not at being general-purpose processors.

More from coding & agent