SemiAnalysis: Vera Rubin + SGLang hits 6.6x per-chip throughput, 4.23x perf/dollar vs GB300
NVIDIAAI · x · 2026-10-14
SemiAnalysis reports Qwen3.8 Flash Next running on NVIDIA Vera Rubin NVL72 with SGLang reaches 4.23x perf-per-dollar versus GB300 NVL72. SGLang's numbers with NVIDIA Dynamo: at matched per-user interactivity (P90 180 tok/s/user, FP4), up to 6.6x total token throughput per chip (128.7k vs 19.4k tok/s/chip) and up to 76% lower modeled cost per token, based on SemiAnalysis hyperscaler-ownership TCO modeling from early-access interpolated previews.
More from Infra
- NVIDIA commits $250M compute over 5 years for UK AI safety, as NHS and banks race to deploy — heyshrutimishra · 2026-10-14
- ECDSA.fail quantum challenge hits 893 qubits x 2.2M Toffolis, 25% cheaper than Google's best — jedisct1 · 2026-10-14
- Llamaforge v0.16: a control panel for upstream llama.cpp with local agents whose quotes are code-verified — Sleepybear2611 · 2026-10-14
- Questioning OpenAI's Decisions API: are cloud round-trip latency and privacy viable in production? — WhichCardiologist800 · 2026-10-14
- Kev-4B Decision Model Goes Live in Beta on Phala with NVIDIA Confidential Computing — bgmshana · 2026-10-14
- Zuckerberg: Meta committed hundreds of billions on AI infra before knowing it would work — GaryMarcus · 2026-10-14