Baseten's WavePP: up to 2.9x prefill throughput on same GPUs, same kernels
baseten · x · 2026-11-20
Baseten introduced WavePP, a pipeline-parallelism (PP) prefill runtime built on TensorRT-LLM. Each PP stage runs as an independent lightweight model worker with its own KV cache and request prep; WavePP overlaps request admission with pipeline execution — asynchronously finding a prefix reusable across all stages, protecting cached state, reserving space for remaining input, and dynamically planning chunk sizes to maximize pipeline fill.
Results:
- Up to 2.91x prefill throughput on GLM-5.2 (PP4) and 2.02x on MiniMax M2.7 under high prefix reuse, beating the TensorRT-LLM baseline in 37 of 40 settings.
- Highest measured throughput in all 18 Kimi K3 settings at concurrency ≥8 (out of 28).
- Same kernels, same GPUs — the gains come purely from runtime scheduling.
More from Infra
- AMD denies short-seller report that sent stock down 13%, calls claims misrepresentations — BenBajarin · 2026-11-20
- RTX Pro 6000 listed at $8000 on eBay, buyer-beware warning — QuanstScientist · 2026-11-20
- First rendered videos from Huawei Ascend GPU surface online — GabGarrett · 2026-11-20
- NVIDIA shows how to build training-ready datasets in two prompts with Encord and Voxel51 — NVIDIAAI · 2026-11-20
- 8-minute music film made 100% locally in ComfyUI on a single Mac, consistent faces throughout — Mardoslazar · 2026-11-20
- Spiral open-sources t4, a Rust object store cache that queries disk at memory speed — sh_reya · 2026-11-20