NVIDIA's 120B MoE Nemotron 3.5 Super VL VLM arrives on SageMaker JumpStart
AWS ML Blog · rss · 2026-11-10
NVIDIA Nemotron 3.5 Super VL, the vision-language model of the Nemotron 3.5 family, is now deployable via Amazon SageMaker JumpStart, accepting images, video, and text with up to 1M-token context.
Architecture
- 120B total / 12B active hybrid MoE backbone: 88 layers interleaving Mamba-2 and MoE layers (512 routed experts, 22 active per token) plus 8 attention layers, with a shared Multi-Token Prediction head for speculative decoding.
- A C-RADIO vision encoder handles images and video frames; each 1536×1024 page image costs 1,536 tokens. Training spans continued pre-training, long-context and software-engineering mid-training, two-stage multimodal SFT, RL with verifiable rewards and human feedback, and multi-teacher on-policy distillation.
Two variants
- NVFP4 (75 GB): quantization-aware distillation from BF16, W4A4 on Blackwell GPUs; default ml.g7.24xlarge with 4× RTX PRO 4500 Blackwell.
- BF16 (249 GB): full-precision reference weights for SFT/RL/distillation/custom quantization, on ml.p5en.48xlarge with 8× H200. Neither variant serves LoRA adapters.
Benchmarks and performance
- NVFP4 essentially matches BF16: identical 62.82 on MMLongBench-Doc, slightly better on OCRBench v2 (68.34) and OSWorld-Verified (52.30 vs 51.14), marginally lower on CharXiv reasoning (79.88) and Video-MME v2.
- Measured on a generate profile (250 in / 1,500 out tokens, 5 concurrent streams): NVFP4 hits 63.07 tokens/sec per request on g7.24xlarge, BF16 reaches 140.85 tokens/sec on p5en.48xlarge; time to first token stays around 0.5s, but rises to 12.62s with 10K-token inputs.
Use cases: document intelligence (OCR, charts, multipage docs), GUI grounding for computer-use agents, short-video Q&A, and multimodal agents with tool calling and JSON schema output. Reasoning is on by default and configurable via enablethinking. Licensed under OpenMDW 1.1.
More from Infra
- Polymarket odds: 23% chance Anthropic announces bankruptcy by end of 2029 — Polymarket · 2026-11-10
- Benchmark: Agentgateway costs ~$280/yr at 5000 routes, rivals up to 182× more — bibryam · 2026-11-10
- Anthropic signs $3.4B deal to power AI data centers with natural gas generators, bypassing the grid — Polymarket · 2026-11-10
- NVIDIA's Jacob Loewenstein: token efficiency matters more than price sheets when picking models — togethercompute · 2026-11-10
- AI networking market to hit $278.9B by 2027 at 95% CAGR, TrendForce forecasts — Beth_Kindig · 2026-11-10
- Peter Diamandis: Data center backlash is the best thing to happen to AI — PeterDiamandis · 2026-11-10