Post-Trained NVIDIA Nemotron Beats Claude Opus in Legal Agent Tasks
ctnzr · x · 2026-08-12
Harvey, in collaboration with Trajectory Labs, post-trained the NVIDIA Nemotron 3.5 Lightning model on the Legal Agent Bench (LAB), achieving remarkable results:
- Performance Leap: Agent performance on held-out LAB tasks jumped from 0% to 8.3%, outperforming both Claude Opus 4.6 (6.6%) and the significantly larger Nemotron 3 Ultra.
- Broad Improvement: Performance improved across nine practice areas without regressions.
- Cost Efficiency: Post-training reduced average model output from 90k to 37k tokens, boosting the reward-per-token by 2.4x.
Trajectory Labs notes that continual learning relies on cheaper retraining loops. While large models might run this loop every few weeks, smaller models like Nemotron 3.5 Lightning can be fine-tuned nightly, per customer, or even per legal matter, bringing metered intelligence closer to reality.
Related event: Fine-tuned NVIDIA Nemotron 3.5 Outperforms Claude Opus(4 posts)→
More from coding & agent
- Everything crashes, but Omarchy sends crash reports to Claude in 1 click — indragie · 2026-10-16
- Six-month prediction: hiring for generic agent harnesses built from scratch will cool — yi111 · 2026-10-16
- Omarchy ISO wouldn't boot on NVIDIA, so he had Claude patch it — indragie · 2026-10-16
- Dev uses Grok to clean up 3D printer firmware, catches it sneaking in nonsense code — yacineMTB · 2026-10-16
- A rude AI agent took 89% of a shared resource in 102-run multi-agent personality experiment — Brave-History-6502 · 2026-10-16
- Guardrails can make risky delegation work — but it takes a full-time team, says iandanforth — iandanforth · 2026-10-16