FP8 + DSpark stack delivers ~2x speedup for Nemotron 3.5 Super VL on H100s
vllm_project · x · 2026-10-15
vLLM project highlights Red Hat AI's two open optimized checkpoints for NVIDIA Nemotron 3.5 Super VL (a 120B multimodal MoE for long-running agents): FP8 via LLM Compressor boosts throughput under load, while DSpark via Speculators improves per-user streaming speed. Benchmarks on 4× H100 show the optimizations complement each other, stacking quantization and speculative decoding for close to 2x speedup with accuracy intact across HumanEval, math reasoning, and RAG workloads. Both checkpoints are open on Hugging Face.
More from Infra
- New inference engine 'e' hits 10,500 tok/s on a single DGX Station — brandon_galang · 2026-10-16
- SGLang ships Day-0 support for FLUX 3 Image Klein: 10 reference images, BF16/FP8/NVFP4 variants — ying11231 · 2026-10-16
- NVIDIA's Nemotron 3.5 Super VL hits CoreWeave: 12B-active multimodal MoE at $0.25/M input tokens — wandb · 2026-10-16
- Chip equipment beats bull case: 2026 WFE seen at $153B, $278B by 2030 — BenBajarin · 2026-10-16
- Steam now launches in 5.23s, beating Epic's 5.47s — and vLLM gets roasted for slow startup — AlpinDale · 2026-10-16
- Shadeform to Announce Six New Features on Oct 19: Lower GPU Costs, More Suppliers — _jaydeepkarale · 2026-10-16