Nemotron Reproduction Still Unverified
JJitsev · x · 2026-07-13
This reply emphasizes that it remains uncertain whether Nemotron 3 Nano has actually been reproduced until the numbers are fully verified.
- Currently, soofi's scores are slightly higher than Nemotron 3 Nano, making it appear like a successful reproduction on the surface.
- However, if there are discrepancies in NVIDIA's original evaluation numbers, the comparison loses its baseline, rendering the conclusion invalid.
- In other words, the core of this debate isn't about "which is better," but rather whether the evaluation metrics are aligned and the numbers are reliable.
Related event: Controversy Over Nemotron Reproduction and Custom Benchmarks(4 posts)→
More from Research
- AI veteran Thomas Dietterich: stop using cs.AI as your primary arXiv category — tdietterich · 2026-10-27
- arXiv advice from T. Dietterich: wrong primary category will almost surely get your paper on hold — tdietterich · 2026-10-27
- arXiv has 4,000+ papers on hold amid submission surge; moderator shares how to avoid it — tdietterich · 2026-10-27
- Community-trained MOSS Sharpness Slider LoRA: 72 photos, 500 steps, +/-1 focus control — aoleg77 · 2026-10-27
- CoreWeave, You.com and NVIDIA lift BrowseComp 22.9% in 3-week RL post-training sprint — RichardSocher · 2026-10-27
- Open-source decision models sold their own base in Red Alert 2 — then learned until no human could beat them — heathercmiller · 2026-10-27