Post-trained NVIDIA Nemotron-3.5-Super-VL hits 35% on AutoResearchExam, matching a 20× larger model
AlexGDimakis · x · 2026-10-16
A UCSD team (Dimakis et al.) with NVIDIA post-trained Nemotron-3.5-Super-VL into an autonomous ML researcher, lifting AutoResearchExam scores from 28% to 35% — matching Qwen 3.8 Max (35.3%), a model with 20× the total parameters, and well above Muse Spark 1.3 (25.8%).
- Tasks: long-horizon, open-ended AI/ML research — building a faster executor for a transformer compute graph, training RL control policies, creating new uncertainty quantification methods; the model submits once and never sees its score until the end.
- Self-context distillation: solve problems with hints, then SFT on those trajectories without the hints — worth +11% alone.
- Reflection: the model reviews its past attempts, writes concrete lessons learned, and SFT on those pushes the score to 0.358.
A representative case of small-model-plus-data-engineering closing the gap with much larger models on research tasks.
Related event: NVIDIA Nemotron Post-Trained as Autonomous ML Researcher(2 posts)→
More from Models
- NVIDIA HF listing for Qwen3.8-Flash-Next-FP4-Local leaks unannounced Qwen3.8 model — TheZachMueller · 2026-10-16
- Muse Spark Open, billed as the most capable US open-weight model, lands on Nebius day one — Arindam_1729 · 2026-10-16
- Model at capacity for a straight week drives frustrated dev back to Opus — joonasvirtanen · 2026-10-16
- Sentry CEO on expensing Claude subs: it's allowed, employers just want ZDR — zeeg · 2026-10-16
- Decision models match LLMs on routing and safety — and are 7-8x faster — randal_olson · 2026-10-16
- Xiaomi MiMo V2.6 Pro takes #1 open-model spot on Agent Arena at $0.11/task — arena · 2026-10-16