SemiAnalysis breaks down NVIDIA LPU's three disaggregated inference configurations
SemiAnalysis_ · x · 2026-09-01
SemiAnalysis outlines three disaggregated (prefill/decode split) inference configurations supported by NVIDIA LPU:
- Rubin Prefill + LPU Decode: fastest interactivity;
- Rubin Prefill + Rubin Decode Attention + LPU Decode FFN: covers the middle of the latency curve;
- Rubin Prefill + Rubin Decode Verification + LPU Drafter: speculative decoding, for the middle-left of the curve.
For low-interactivity workloads, raw Rubin still wins. They look forward to Rubin + LPU performance curves on open-source agentic benchmarks like AgentX.
Related event: NVIDIA LPU Supports Three Disaggregated Inference Modes(2 posts)→
More from Infra
- RL train-sample systems are frontier AI's hardest infra problem — and a trade secret — brianryhuang · 2026-10-16
- Local tuning of GLM-5.3-Flash on 2×GB10 lifts decode from 49 to 58 tok/s — EAccelerate_42 · 2026-10-16
- 3x DGX Spark cluster runs GLM-5.3-Flash locally behind a proxy, 1.75M tokens/day at 119W — Teknium · 2026-10-16
- Optuna creator Akiba: when 2 hyperparameters cost $20M, what's left for BO/HPO — Stefania_druga · 2026-10-16
- Amazon Proposes Per-Request Validated Evidence Caching for Production RAG — _reachsumit · 2026-10-16
- LinkedIn's TransX report: sequence transduction for recsys, +6.02% CTR in A/B tests — _reachsumit · 2026-10-16