Nemotron 3.5 Super VL benchmarks leak: strong agentic scores, weak HLE and long context
testingcatalog · x · 2026-10-15
testingcatalog shared benchmark charts comparing Nemotron 3.5 Super VL (120B-A12B) against open rivals including Qwen3.5/3.8, DS-V4 Flash, GLM-5.3 Flash, and Inkling Small:
- Agentic: SWE-bench Multilingual 67.8, SWE-bench Pro 51.8, Terminal-Bench 2.0 60.7, BrowseComp 76.9, SciCode 47.1 (best in class)
- Reasoning: GPQA Diamond 89.6, slightly below Qwen3.8's 92.7
- Multimodal: CharXiv-RQ 80.8, OCR-Bench-v2 68.1, MMLongBench-Doc 62.8
- Weak spots: HLE without tools at just 31.8; GraphWalks long context 63.3, lowest of the group
More from Models
- OpenAI's Decisions API hits GA: near real-time routing up to 10x faster than GPT-6 Luna — OpenAIDevs · 2026-10-23
- DeepLearning.AI weekly: GPT-6.1 hits 97.9% byte parity on COBOL migration — DeepLearningAI · 2026-10-23
- Marker's lite mode scores 75.1 vs 42.3 for Azure Read on OmniParseBench — VikParuchuri · 2026-10-23
- Datalab launches lite OCR at $0.50 per 1,000 pages, 20-30x cheaper than Azure, Google and AWS layout APIs — VikParuchuri · 2026-10-23
- LiveNerf's 29-day data finds no nerfing of Claude Opus 5.5 — TheOnlyVibemaster · 2026-10-23
- PHP Foundation reveals Anthropic's Claude cyber model scanned PHP, validated with OpenAI's Daybreak Red — saibharadwaj · 2026-10-23