OAT: Finding Failed Steps via Successful Trajectories
Samuel Yeh · hf · 2026-07-16
This work investigates **failure attribution for LLM agents**: how to identify the exact step that causes task failure within a failed trajectory. - **Existing Issues**: - Prompt-based methods are computationally expensive; - Methods relying on step-by-step annotation of failed trajectories are costly and hard to scale. - **Core Approach**: Proposes **OAT**, an unsupervised method trained exclusively on successful trajectories to locate failed steps during inference. - Formulates the problem as **one-class learning**; - Uses **neural controlled differential equations** to model the dynamic patterns of successful trajectories in latent space; - During inference, assigns anomaly scores to steps in failed trajectories based on their deviation from "success dynamics," forming an error step set. - **Results**: Trained on just **100 successful trajectories**, OAT is **200–5000x faster** than prompting-based baselines while achieving **+20%** and **+7%** F1 improvements on in-domain and out-of-distribution datasets, respectively. - **Significance**: Provides a lightweight, scalable failure diagnosis direction for agent debugging.
Related event: Microsoft Proposes OAT for LLM Agent Failure Attribution(2 posts)→
More from coding & agent
- Autoresearch proposes packaging ML runs as studies with questions, analysis, and code diffs — morgymcg · 2026-07-21
- CHAP defines approvals, handoffs, and audit logs for human-agent workflows — DeliveryTechnical199 · 2026-07-21
- The author says Codex reached 20x and is now debugging spec decoding on a hybrid parallel setup — TheZachMueller · 2026-07-21
- Axcess adds an MCP connector for WCAG accessibility checks that scanners miss — modelcontextprotocol · 2026-07-21
- X post asks whether Cursor Composer, built on Kimi models, would also be banned — max_paperclips · 2026-07-21
- A developer’s Codex usage is draining pooled enterprise credits at a small company — Distinct_Relation_62 · 2026-07-21