Nemotron 3.5 Super + Surge RL matches GPT-5.6 Terra across 12 benchmarks
echen · x · 2026-11-06
Surge AI trained NVIDIA Nemotron 3.5 Super (120B total/12B active) with Thinking Machines' Tinker on four RL datasets spanning software engineering, instruction-following, long-horizon enterprise workflows, and multimodal work.
- Gains of 4.5–18.6pp across all 12 benchmarks (GDP.pdf +18.6, HANDBOOK.md +12.8, AutomationBench +12.7, Terminal-Bench 2.1 +6.4)
- Matches or beats much larger closed models: GPT-5.6 Terra on SWE-Bench Pro V2, Claude Opus 4.7 on Terminal-Bench 2.1, and 744B GLM-5.2 on AutomationBench
- AutomationBench end-to-end completion nearly doubled, 15% → 28%
- Learned habits (evidence-checking, code testing, multi-step completion) generalized to unseen settings
Takeaway: high-quality RL data makes frontier-level performance accessible to open models.
Related event: Surge AI Trains Nemotron 3.5 Super with RL on Tinker, Nearing GPT-5.6(5 posts)→
More from Models
- Ripple-65B Preview: 65B Dense Model Claims Open-Weight SOTA on Agentic Coding — annadgoldie · 2026-11-06
- Greg Brockman Endorses GPT-6 Cyber, Security Researcher Calls It Light-Years Ahead — gdb · 2026-11-06
- Gemini in Volvo cars flunks basic tests: refuses windows, offers 'friendship warmth' — giffmana · 2026-11-06
- Community probe: original gpt-oss knows nothing of the 'hidden classifier' fine-tune — repligate · 2026-11-06
- Anthropic's next model reportedly finished training; METR eval shows biggest step-change — daniel_mac8 · 2026-11-06
- Knowledge cutoffs aren't a hard date — model accuracy drops well before it — RexDouglass · 2026-11-06