OAT uses 100 successful trajectories to debug failing AI agents without failure labels
TheTuringPost · x · 2026-07-22
OAT flags failing agent steps without failure training
A new paper proposes OAT, a method for debugging failed AI agents without training on failures. It learns the pattern of successful trajectories and then identifies the steps in a failed run that deviate from the learned path.
Key points:
- Trained on only 100 successful trajectories
- Uses no failure examples and no step-level labels
- Reported to be 200–5,000× faster than prompting-based attribution
- Achieves +20% F1 in-domain and +7% OOD
The main idea is a shift from asking another LLM to explain what went wrong, toward cheap anomaly detection over the execution trace. The paper also notes an important caveat: unusual steps are evidence, but they do not prove causation.
More from coding & agent
- Treating agents like 50 First Dates: a 3-layer context system so every conversation doesn't start from zero — evielync · 2026-09-11
- SmolVM open-sources persistent computer infrastructure for agents that outlive chat sessions — aniketmaurya · 2026-09-11
- ARRM targets silent economic regressions in AI agents that functional tests miss — Beautiful_Belt_601 · 2026-09-11
- Dev builds browser 3D pizza delivery game with Claude: physics, GPS pathfinding, traffic AI — vinishkapoor · 2026-09-11
- Build X Carousel Posts from One Wide Image: A Splitter Tool Plus YouMind Skill Workflow — sujingshen · 2026-09-11
- "Anyone still coding the old way?" The joke capturing post-AI programming culture — lxfater · 2026-09-11