ICML Paper: Adaptive Failure Taxonomy
wzenus · x · 2026-07-11
An ICML FAGEN Workshop paper titled "Fantastic Adaptive Taxonomies and How to Use Them" won the Best Paper Award. The author outlined the core concept: failure taxonomies can transition from static, hand-crafted designs to being dynamically and adaptively generated based on agent weaknesses and tasks.
Problem Addressed
Traditional approaches often use preset failure taxonomies like MAST, but the author argues these fixed taxonomies fail to accurately reflect actual agent behaviors.
Primary Use Cases
The paper demonstrates three applications for adaptive taxonomies:
- As a test-time scaling tool to serve best-of-N judges
- As a mutation feedback mechanism within optimization loops
- As runtime feedback signals for coding agents
Core Methodology
- Discovering failure modes by observing agent rollouts
- Dynamically constructing a classification system better aligned with current tasks and weaknesses based on these observations
- Treating the taxonomy itself as an iteratively optimizable object, rather than a fixed set of labels
Related event: Adaptive Failure Taxonomies Win ICML Workshop Best Paper(3 posts)→
More from coding & agent
- Kolab Benchmark Uses Pro Screen Recordings to Show AI Agents Overestimate Collaboration Quality — minnesotanlp · 2026-10-27
- Dev argues Claude burns far more tokens than GPT models for the same coding work — haider1 · 2026-10-27
- Stop building agents that can't change models: stable contracts, external state, real evals — rseroter · 2026-10-27
- Cheap models can cost more in agent harnesses than frontier ones, researcher warns — vasuman · 2026-10-27
- Kanzo: A 5k-Line Rust Kanban Binary With MCP for Claude Code and Codex — allmudi · 2026-10-27
- 'A human approves everything' is a trap: approval fatigue and how to actually design safe agents — Efistoffeles · 2026-10-27