NVIDIA Nemotron 3.5 scales LLM self-verification, +9-10 points on agentic benchmarks
sarahcat21 · x · 2026-10-31
NVIDIA's latest release adopts LLM-as-a-Verifier: scaling self-verification with Nemotron 3.5 Super lifts agentic benchmarks — Terminal-Bench 2.1 from 63% to 72% (+9%), DeepSWE from 25% to 35% (+10%). Commentator sarahcat21 argues automated verification is the future but discrete scores aren't the answer; she's curious about scaling verifiers along other dimensions, including decomposed evaluation criteria.
More from Models
- Perplexity early evals: Gemini 4 Carbon tops WANDR benchmark for long-form research — AravSrinivas · 2026-10-31
- DeepSeek's silent downtime upgrade caught by the pelican bicycle test — teortaxesTex · 2026-10-31
- ARC Prize 2026 closes in 3 days as Yi-Chia Chen tops ARC-AGI-3 at 75.56% — GregKamradt · 2026-10-31
- Fable 5.5 Max stuns with gorgeous SVG of a pelican riding a bicycle, headed to Claude Code — ChrisGPT · 2026-10-31
- OpenAI reportedly to train on users' Daybreak sessions with no opt-out — dyn___ · 2026-10-31
- Mistral Large 4 scores 94% on 1,273 USMLE-style questions but flips 44 at temp 0 — MaziyarPanahi · 2026-10-31