Pipeline parallelism beats TP2 and DEP2 by 23% throughput for agentic serving on MiniMax-M3.1-Flash
woosuk_k · x · 2026-11-17
inferact argues pipeline parallelism (PP), rarely chosen for prefill due to pipeline bubbles, makes sense for agentic serving.
- On MiniMax-M3.1-Flash with vLLM on two NVIDIA B300s, PP2 sustained 23% more requests per GPU than DEP2 and 17% more than TP2 at a 2s p50 TTFT target.
- Benchmarks show TP2 has the lowest p50 TTFT up to 2.8 req/s per GPU; above that PP2 wins. Each layout ran three 60s Poisson runs (seeds 42/43/44), medians joined in the curve.
More from Infra
- SpaceX pushes FCC for last 50 MHz used by planes and cars ahead of Nov. 19 vote — RachelVT42 · 2026-11-17
- Hadrian invests $300M to upgrade an 1866 Pennsylvania foundry as its Factory 6 — oyhsu · 2026-11-17
- Harvard study: hyperscale data centers used nearly 2% of US electricity in 2024-2025 — Polymarket · 2026-11-17
- Qwen 3.8 27B at 300-500 tok/s on a 5090 laptop, tuned for parallel agents — tebjan · 2026-11-17
- Stanford's Azalia Mirhoseini predicts a Cambrian explosion of model-tailored AI chips — agihouse_org · 2026-11-17
- GPU profiling webinar: PyTorch recompile spikes and a claimed 46% throughput gain — cfregly · 2026-11-17