Cerebras pushes disaggregated inference: heterogeneous prefill pools feeding wafer-scale decode
ai · x · 2026-10-16
Cerebras CEO Andrew Feldman explained the benefits of disaggregated inference: the company uses heterogeneous prefill pools feeding into wafer-scale engines for decode, scaling token speed independently of batch size. The thread also highlights d-Matrix's layer-level split — 1x H100 + 8x Corsair XPUs disaggregating attention (memory-bound) from FFN (compute-bound) to hit 845 tok/s per user.
More from Infra
- Muse-Spark open-weights drop: 816B-A42B MoE model gets 359GB MLX 3-bit build — ns123abc · 2026-10-16
- vLLM ships Day-0 LoRA RL for Muse Spark Open, lifting AIME from 0.56 to 0.85 — vllm_project · 2026-10-16
- DSpark draft model for Muse Spark Open proposes 7 tokens per pass with 23-42% acceptance — vllm_project · 2026-10-16
- Cloudflare Stream's WebRTC ultra-low-latency live streaming is now Generally Available — ritakozlov · 2026-10-16
- YC-backed Overshoot launches inference optimization, claims $5M saved for AI teams — ycombinator · 2026-10-16
- Meta's 816B open-weight Muse Spark Open hits Groq preview at 350+ tokens/sec — GroqLLC · 2026-10-16