NVIDIA paper predicts post-training coding-agent performance from base models with ρ=0.964
sudoraohacker · x · 2026-10-20
NVIDIA researchers published "Before They Can Solve: Predicting Post-Training Coding-Agent Performance from Base Models" (arXiv:2610.10478), tackling a core question: which base checkpoint deserves an expensive round of agentic post-training?
- Motivation: end-to-end pass@$K$ fits agentic coding poorly — many base checkpoints can't reliably produce well-formed tool calls; single-shot tasks sidestep multi-step tool use entirely
- Method: treat successful post-trained agent trajectories as a lookahead signal — replay each trajectory and rerun tests after every code-changing step to identify the decisive step that flips the repo from failing to passing
- Result: predicts future SWE-bench rankings with ρ=0.964
- The team says the method is already used in Nemotron mid-training (the paper behind Super 3.5)
Related event: NVIDIA Team: Predicting Coding-Agent Performance Before Post-Training(9 posts)→
More from coding & agent
- OpenCode hacked: video surfaces on security breach of open-source coding tool — RelevantEmergency707 · 2026-10-20
- Stanford researchers study cascade harnesses to maximize agent performance at low cost — james_y_zou · 2026-10-20
- Dev ran a Shopify store end-to-end with Hermes-Agent, making physical merch — alexcovo_eth · 2026-10-20
- LlamaIndex launches ExtractBench Pro: long-tail document extraction that stumps frontier models — llama_index · 2026-10-20
- vLLM ecosystem ships d3-turbo: multimodal decision model, 15x faster, tops Jev Decision Index — vllm_project · 2026-10-20
- Compacting a compacted LLM context: engineers' silent pain — nrehiew_ · 2026-10-20