Lambda benchmarks Nemotron 3.5 Super VL: one NVFP4 Blackwell GPU out-throughputs two in BF16

TheZachMueller · x · 2026-10-15

Lambda published deployment benchmarks for NVIDIA's Nemotron 3.5 Super VL (120B total / 12B active, native vision, 1M context) on SGLang:

NVFP4 on 1× Blackwell: 1,860 tok/s peak throughput (C16), 587 tok/s/user single-stream
BF16 on 2× Blackwell (TP2): 1,669 tok/s peak, 530 tok/s/user
On the same 8,192-in/1,024-out workload, a single NVFP4 GPU beats two BF16 GPUs, roughly 2.2× per-GPU throughput, since Blackwell runs FP4 natively
Both paths use EAGLE speculative decoding (5 steps, 6 draft tokens) via multi-token prediction
Caveat: CUDA graphs are captured only up to batch size 16, so throughput regresses beyond C16

Takeaway: native FP4 inference lets one card replace two, cutting deployment cost significantly.

More from Infra