NeurIPS paper OAT: attribute agent failures by learning only from success flows
SharonYixuanLi · x · 2026-10-24
Sharon Yixuan Li introduces the NeurIPS 2026 paper "Tracing Agentic Failure from the Flow of Success," presenting OAT, a lightweight failure-attribution method for LLM agents.
Problem: Debugging agentic systems hinges on locating which step broke a run, not just detecting failure. Existing options are costly: prompt-based pipelines have an LLM re-read entire trajectories (slow, expensive), while post-trained attribution models need painful step-level failure labels.
OAT's idea: Learn only what success looks like—no failure labels required. Training uses Neural Control Differential Equations to model and reconstruct hidden representations of successful trajectories; at inference, OAT predicts the expected successful path for a failed run and flags failure-contributing steps via anomaly scores from the distance between actual and expected representations.
Authors: Samuel Yeh, Yiwen Zhu, Shaleen Deep, Sharon Li (UW-Madison, Microsoft Research).
More from coding & agent
- Free LLM Dojo launches: 50+ interactive labs and 490 AI-engineering interview questions — sharpeye_wnl · 2026-10-25
- mlx-decision runs decision models locally on Apple Silicon, up to 6x faster than PyTorch — armbues · 2026-10-25
- How do teams share memory across multiple AI agents? — belhelha · 2026-10-25
- Vibe Monitor ships after 41 beta iterations: local stdio MCP adds architecture checkpoints to coding agents — Either-Alternative32 · 2026-10-25
- KNOWS benchmark: best computer-use agent fully solves just 3.6% of tasks — anmarasovic · 2026-10-25
- Dev feeds Grok thermal camera and live video to auto-debug overheating hardware — Baconbrix · 2026-10-25