Nvidia paper: replaying agent trajectories predicts which base models are worth post-training

rohanpaul_ai · x · 2026-10-12

A new Nvidia paper (arXiv:2610.10478) tackles how to predict which base checkpoint is worth an expensive round of agentic post-training. Key points:

End-to-end pass@K is a poor fit for agentic coding: many base models can't reliably produce well-formed tool calls; short-horizon tasks sidestep this but skip the core multi-step capability.
Method: treat successful post-trained agent trajectories as a lookahead signal — replay each trajectory and rerun tests after every code-changing step to find the "decisive step" whose cumulative patch flips the repo from failing to passing.
Finding: scoring base models on every step of an agent run works worse than scoring only the first edit that makes the tests pass.

Authors include Bryan Catanzaro, Mostofa Patwary, and other Nvidia researchers.

More from coding & agent