NVIDIA details 5 co-design levers as Vera Rubin NVL72 hits 30x throughput per watt vs GB300
NVIDIAAI · x · 2026-11-06
NVIDIA's technical blog breaks down five system-level co-design techniques for maximizing every byte in the AI factory:
- NVFP4 quantization: shrinks model footprint 3.5x vs FP16 and 1.8x vs FP8 with accuracy preserved, freeing HBM for KV cache, larger batches, and higher concurrency
- Wide expert parallelism: TensorRT-LLM spreads MoE experts across 8+ GPUs, raising arithmetic intensity via weight reuse
- Dynamo + KV Cache Runner: treats KV cache as a fleet-wide resource with KV-aware routing, disaggregated serving, and NIXL-based tiering across GPU, CPU, and storage
- Vera CPU: 88 Olympus cores (1.8x single-thread), 1.5TB LPDDR5X coherently linked to Rubin at 1.8TB/s, staging idle KV cache during tool calls
- CMX storage: pod-level G3.5 tier on BlueField-4 STX, Spectrum-X, and DOCA for cross-node KV cache sharing
Result: on the same 288GB per GPU, Vera Rubin NVL72 delivers up to 30x the throughput per megawatt of GB300 NVL72.
Related event: NVIDIA Details Five System-Level Ways to Push Past the AI Memory Wall(3 posts)→
More from Infra
- Chip designer pitches Apollo on small AI data centers across 20 US states — pstAsiatech · 2026-11-06
- SpaceX data center pledges 40% power discounts and billions in community benefits — Scobleizer · 2026-11-06
- Mystery seller seeks to dump $500 million of data center loans — dinabass · 2026-11-06
- Collinear AI runs full-scale agentic RL on Together Post-Training with self-onboarding — togethercompute · 2026-11-06
- SemiAnalysis: Google TPUv7 Delivers Up to 50% Better Performance-per-Dollar Than B200/B300 for Inference — vllm_project · 2026-11-06
- Meta President: new US data center build-outs will be 'very challenging' — SumitGup · 2026-11-06