Fine-tuned Nemotron 3.5 lifts hard Verus proof pass rate from 43% to 73%
fhuszar · x · 2026-10-15
The ReasonableIO team tested whether NVIDIA's open-weight Nemotron 3.5 Super VL can write challenging Verus proofs for TLA+ specs — and found it can. With novel datasets and fine-tuning, pass rates on the hardest benchmark jumped from 43% to 73%. Their report covers dataset construction, the fine-tuning process, and bridging formal verification to real-world software.
Related event: Fine-Tuning Nemotron on 3,000 Verus Proofs Boosts Pass Rate to 73%(2 posts)→
More from Research
- Intelligence is proportional to data: reorder tasks by data volume and the jagged frontier becomes a spiral — sebkrier · 2026-10-16
- Doc2Harness: New benchmark finds best coding agent passes only 53% across 329 cases — _akhaliq · 2026-10-16
- Stanford's Fei-Fei Li team open-sources OpenWAM, a unified world-action model framework for robots — jiqizhixin · 2026-10-16
- NVIDIA's TacitVLA: robots reason without reading their own thoughts, 3.4x lower latency — arankomatsuzaki · 2026-10-16
- Cohere Labs world model course session 4: DreamerV1-V4 architectures in one go — Cohere_Labs · 2026-10-16
- GPT-6.1 Sol audit finds 6% of arXiv papers had false main theorems, down to 1.5% recently — nitarshan · 2026-10-16