AMD and Cerebras Split LLM Inference: KV Cache Bridges the Hardware Gap

TheTuringPost · x · 2026-08-02

AMD and Cerebras are building a novel AI infrastructure that splits LLM inference into two phases across different hardware architectures:

AMD Helios handles the prefill phase, reading the prompt and building the KV cache.
The KV cache and request metadata are then transferred to a Cerebras CS-3 wafer-scale engine.
Cerebras uses this cached state to generate response tokens at extremely high speeds.

This heterogeneous combination is transparent to the user but significantly boosts data center efficiency. The companies claim up to a 5x improvement in tokens per second per watt. However, the latency of transferring the KV cache across systems will be the biggest potential bottleneck for this architecture.

Original post →

AMD and Cerebras Split LLM Inference: KV Cache Bridges the Hardware Gap

More from Infra