SGLang MegaKernel + PD Pushes the TCO-Latency Frontier on GB300, SemiAnalysis Benchmarks Show
AccBalanced · x · 2026-10-27
SemiAnalysis congratulates the SGLang and RadixArk teams on MegaKernel + PD, which improves both high-concurrency throughput and low-latency serving. The accompanying SemiAnalysis InferenceX chart benchmarks DeepSeek V4.1 Flash 552B agentic total tokens per $1 TCO vs P90 interactivity on GB300 NV72 across DEP/TEP/TP topologies, showing the new configs pushing the cost-latency Pareto frontier well beyond prior setups (GB300 TCO: $2.31/chip/hr).
More from Infra
- Reflection AI's Beam-1.0 tops US open-weight models, hitting 88% of Qwen3.8-2.4T with 23B active params — ryanshrout · 2026-10-27
- Re-uploading a 239GB robot dataset moved only 2.6MB — inside a petabyte-scale data pipeline — mervenoyann · 2026-10-27
- Intel to acquire Classiq to build full-stack quantum computing platform — BenBajarin · 2026-10-27
- Ant's own data centers and models on in-house inference silicon will boost margins — BenBajarin · 2026-10-27
- Grand theft AI: Highway thieves steal millions in data center gear — supertrope · 2026-10-27
- Sail equivalence proof and riscv-coq cross-checks uncover real RISC-V spec bugs — xennygrimmato_ · 2026-10-27