NVIDIA and Nebius benchmark Cosmos 3 Super video serving: latency vs throughput on HGX B200/H200
NVIDIA Developer · youtube · 2026-09-24
NVIDIA Developer and the Nebius Physical AI team break down serving world models for video generation, where latency/throughput tradeoffs differ sharply from LLMs. They benchmark NVIDIA Cosmos 3 Super via vLLM-Omni on HGX B200 and H200 across four topologies (one 8-GPU replica to eight single-GPU replicas), showing how replica count, parallelism and concurrency affect latency and validated video-seconds per node-hour — including why faster per-request responses can mean less total output per node. Benchmarks are reproducible from Nebius's open-source repo.
More from Infra
- GPU Rental Prices Hit Record Highs: B200 at $6.97/hr, H100 at $3.06 — inductionheads · 2026-11-11
- A Datacenter in Every Office: Building Work-Safe Local AI Racks — dee_hw · 2026-11-11
- "Compute is about power": the AI race's real bottleneck is energy, not chips — _sholtodouglas · 2026-11-11
- CoreWeave CEO says he canceled his 10b5-1 plan, halting insider selling — firstadopter · 2026-11-11
- UK chip start-up Fractile hits $7bn valuation, per FT — evisoft · 2026-11-11
- Ubuntu hit by major outage, Canonical status page confirms — conannn · 2026-11-11