Evidence-guided recovery helps AI agents handle ambiguous tool outcomes, beating prompting by 36 points
Genliang Zhu · hf · 2026-11-13
This paper tackles a real agent pain point: tool timeouts leave agents facing ambiguous outcomes and partial execution — an action may have failed, committed, or remained in flight while other workflow steps already completed.
- The authors formalize evidence-guided task recovery over approved obligations, provider capabilities, and states compatible with public evidence.
- A shared admissibility check handles key expiry, authorization timing, dependencies, and residual effects; a terminal certificate separates actual goal satisfaction from publicly supported safe completion.
- Experiments: planning certified 28 of 37 assignments; in a preregistered comparison, planning advice certified 30 completions versus 9 for evidence prompting across 58 paired units — a 36.2-percentage-point gap (p=0.0078) with fewer proposals (272 vs 399).
- All 116 trajectories pass independent replay; guarantees tie to declared provider capabilities rather than assuming third-party exactly-once execution.
More from coding & agent
- OpenAI Codex CLI 0.169.0 ships multi-tab side chats, task search, and custom provider onboarding — github-actions[bot] · 2026-11-13
- Three quotumorg Hermes plugins land in NousResearch marketplace: precedent rulings and exfil blocking — Teknium · 2026-11-13
- Matt Pocock wants every important PR to ship with a demo video, building an /animatic skill — mattpocockuk · 2026-11-13
- HiVe-Mem: NTU-led framework for multimodal agent long-term memory — NanyangTechnologicalUniversity · 2026-11-13
- ArtCraft clones Photoshop with AI as open source, has shipped 17 tools and wants to 'rebuild everything' — altryne · 2026-11-13
- Solo dev ships logic-puzzle roguelike built with Claude that screenshot-tests itself — TheCodingTutor · 2026-11-13