Agent Error Dataset: 50,000 Error-Diagnosis Pairs Boost Pass Rate to 51.1%
illinois · hf · 2026-10-01
The Agent Error Dataset (AED) provides 50,228 error-diagnosis pairs from 9,961 tasks across 33 environments, 19 harness families and 23 policy models. In 3,062 matched replays, first-proposal corrections raise verifier pass rates from 18.4% to 51.1%; fine-tuning on diagnosis data lifts Qwen3-8B's exact-step agreement from 47.2% to 63.6%, beating the strongest prompted baseline at 54.7%.
More from coding & agent
- Priors: An Onchain Credit Bureau for AI Agents Emerges — econoar · 2026-10-01
- Tripo API integrates with Unbound for real-time sculpting of AI-generated 3D assets — yshan2u · 2026-10-01
- Veteran game dev lists 10 ways AI saves time: crash logs, CMake, 400ms frame hitches — draginol · 2026-10-01
- NVIDIA's Mid-Harness Scales Actions at the Model-Harness Boundary, Lifting TerminalBench Pass@1 to 68.03% — nvidia · 2026-10-01
- Amazon's SMART Self-Evolving Multi-Agent System Tops All 15 Subtitle Arena Directions, Cuts Penalty 6.9% — amazon · 2026-10-01
- Gary Bernhardt hits all-time low faith in AI agents: they "fix" tests by deleting them — sidjustice_ · 2026-10-01