AdaMAST: Automating Agent Failure Taxonomies Boosts SWE-bench to 70.7%
berkeley_ai · x · 2026-08-01
Researchers from UT Austin released a new paper on AdaMAST (Adaptive Failure Taxonomies), introducing a method to automatically categorize how and why AI agents fail.
- Core Mechanism: It learns system-specific failure taxonomies from raw execution traces and reuses them across runs to identify exactly where, when, and how agents fail.
- Performance: When integrated as a Claude Code skill, it improved the SWE-bench Verified Mini score from 64.0% to 70.7%.
- Benchmarks: On TerminalBench 2, it achieved 89.9% when using AdaMAST-generated taxonomies with an Opus 4.6/Forgecode harness and a Best-of-N Judge.
Related event: AdaMAST Automates Agent Failure Classification(2 posts)→
More from coding & agent
- 'LLM code is sloppy' rebutted with real ffmpeg code: identical if/else branches in a top library — kingslyj · 2026-10-16
- Office-editing tool calls need file, worktree and unit: lessons from dsh-univer-office — FunDistribution6537 · 2026-10-16
- Architect MD workflow: Gemini one-shot outputs that beat Opus, via 5-lens plan critique — oikk01 · 2026-10-16
- Prove a coding agent actually ran the tests: check artifacts, not approvals — ComprehensiveShake76 · 2026-10-16
- Building an acceptance harness for AI spreadsheet edits: allowlists, recalc checks, idempotency — ukdicha-modakkk · 2026-10-16
- Agent actions need checks outside the model: NVIDIA demos lean on guardrails amid wave of real agent incidents — Creamy-And-Crowded · 2026-10-16