NVIDIA + RadixArk boost SGLang Rubin performance up to 10x in under 5 days
ying11231 · x · 2026-11-12
SemiAnalysis InferenceX preview data shows NVIDIA, partnering with RadixArk, improved SGLang performance on Vera Rubin by up to 10x in under 5 days. Benchmarked on Qwen3.8 Flash Next 176B agentic tokens per $1 TCO vs P90 latency: 12.9x at low latency, 1.59x mid-range, 2.49x at higher latency, with KV offload enabled peaking near 620M tokens/$TCO. VR200 hyperscaler-volume ownership cost listed at $3.61/chip/hr. Results are official preview and may change.
More from Infra
- Together Compiler: agent-driven continual compilation for AI inference — togethercompute · 2026-11-12
- Kaspersky details Black Owl APT breach of software vendor eyeing supply-chain attack — cyb3rops · 2026-11-12
- Speculative decoding draft models for Qwen3-8B compared: WhiFlash vs Tengyunw benchmarks — maxsloef · 2026-11-12
- AMD runs MolmoAct2 robot policy locally on a Ryzen AI Max+ 395 mini PC — DJiafei · 2026-11-12
- Opinion: AI infra is becoming a coding runtime as intelligence units outgrow tokens — prateekj · 2026-11-12
- PrefixRelay: open-source proxy gives Cursor 97%+ KV cache hits on local models — jjusko20 · 2026-11-12