Nvidia paper: scoring the decisive passing edit predicts agent post-training payoff at 0.964
rohanpaul_ai · x · 2026-10-12
- Nvidia paper proposes replaying successful agent trajectories to find the first edit that makes tests pass, then scoring base models on that decisive step.
- Three screens—BPB, Patch MCCQ, prefix-conditioned pass@K—ranked 10 model pairs in agreement with post-trained SWE-bench Verified pass@1 at 0.964.
- Practical takeaway: before paying for expensive post-training, run this check with only a benchmark's successful runs and its verifier.
More from coding & agent
- SubQ Snap launches as a long-context decision model: 384K tokens, decisions in ~2-3s at $0.05/M — ryanallred · 2026-10-13
- One wrong 'merge it' prompt shipped ts-rust to T3 Code for 8 hours unnoticed — zeeg · 2026-10-13
- Don't specify languages or architecture for new projects — let the model research and choose — sytelus · 2026-10-13
- GPT-6 one-shots a custom IR blaster app to replace a dead AC remote — PrasoonPratham · 2026-10-13
- NVIDIA's NURBS-based CAD kernel SMLIB goes open source, with MCP agent support — burhop · 2026-10-13
- Why checking errors is bad: Jimmy Koppel's contrarian take on defensive coding — jimmykoppel · 2026-10-13