StopLoss: A Cheap External Monitor Predicts Coding-Agent Failures Early, at 10-100x Lower Cost Than LLM Judges
Jiarong Liang · hf · 2026-10-21
Researchers at the University of Waterloo's TIGER-AI-Lab released StopLoss, a paper and open-source project named after the financial stop-loss order. It trains a small external monitor to predict whether a coding-agent run will fail verification, using completed trajectories and their verifier outcomes.
Key points:
- A fine-tuned LM scores recent tool steps in a single forward pass; a threshold calibrated on successful runs turns the score into an abort decision — no changes to the executor agent.
- Trained on GPT-family SWE-rebench trajectories, evaluated on SWE-bench Verified and a SWE-bench Pro subset with GPT and Claude executors.
- Compared against six general-purpose LLM judges reading identical windows, outcome supervision lifts failure prediction from near chance to a useful signal at the first check, at one to two orders of magnitude lower cost per check.
Code: github.com/TIGER-AI-Lab/StopLoss
More from coding & agent
- Research agent Épi sets 120 records, narrows n-queens constant interval 84x — dair_ai · 2026-10-21
- Compaction hooks in practice: keep last n messages verbatim, summarize the rest for full agent continuity — MoonL88537 · 2026-10-21
- The blank prompt box hides real operational burden: context, workflow and supervision fall on users — _AustinCalvert_ · 2026-10-21
- DeepMind paper: self-evolving agents forget as they learn — fastest learner GEPA also breaks old tasks most — rohanpaul_ai · 2026-10-21
- arXiv: Are Self-Evolving Agents Continual Learners? Plasticity comes at the cost of stability — rohanpaul_ai · 2026-10-21
- PM OS vs Team OS vs Company OS: how to layer your AI setup and which to build first — aakashgupta · 2026-10-21