AGENTSCOPE: Microsoft & Tsinghua's neuro-symbolic method pinpoints LLM agent failure steps and types
rohanpaul_ai · x · 2026-09-09
The Microsoft–Tsinghua paper behind the structured agent-run diagnosis result is now on arXiv (2609.02371): "Diagnosing with Insights: Structured Analysis of Agent Failures via Behavioral Abstractions."
Method
- AGENTSCOPE is a neuro-symbolic approach: it abstracts agent trajectories into structured behavioral representations and introduces neural invariants to specify behavior properties.
- LLM-guided reasoning over the structured representation against the invariants pinpoints both the failure step and its failure type.
- Evaluated on the public Who&When dataset and a new, more comprehensive AgentErrata dataset, it significantly outperforms prior SOTA in fault localization and attribution accuracy.
Motivation: traditional software debugging can't handle LLM agent failures, and using LLMs alone as judge yields unreliable results.
More from coding & agent
- New open skill turns folders of PDFs into clean Markdown with an agent swarm — doodlestein · 2026-09-09
- Grok Build adds fully transparent workspace via /theme transparent command — XFreeze · 2026-09-09
- Zig coding agent fx v0.0.8: <2ms cold start, 40x faster init, 6MiB binary — evilrabbit_ · 2026-09-09
- Waterloo dropout turned OpenAI MTS on why computer use is the road to AGI — dhruv2038 · 2026-09-09
- Magnitude open-sources Apple silicon inference server that auto-tunes local models for your Mac — nickbaumann_ · 2026-09-09
- Tencent paper: continuously harder task environments beat co-evolution, +8.6pp on Terminal-Bench — rohanpaul_ai · 2026-09-09