UndoBench: 42% of agent capability gains break recoverability, Git revert isn't enough
AI Engineer · youtube · 2026-11-08
Talk from the AI Engineer CODE summit by Tanmay Sah and Dolly Sah on a capability coding agents lack: undoing their own mistakes.
- Git doesn't cover external API calls, runtime config or notifications, and retrying doesn't reverse side effects — completing a task doesn't mean an agent can put things back.
- In their EvoUndo research, 197 of 478 changes that made an agent more capable (42%) also broke its ability to recover.
- They propose capturing original state, choosing the right response (retry, restore, compensate or escalate), and independently verifying the result.
More from coding & agent
- Qwen builds its own LLM: trains a 100k-parameter model in your browser, and narrates it — victormustar · 2026-11-08
- Agent Lightning paper trains models through the same harness used at deployment — bibryam · 2026-11-08
- 12 years, 8,000 commits: solo dev takes Spartan engine from yellow triangle to drivable island — ssh4net · 2026-11-08
- FFmpeg's Assembly code is the go-to test for whether LLMs can out-code human engineers — sh4dy_0011 · 2026-11-08
- DIY LLM vulnerability scanner finds 10 bugs for ~$20/day, 8 novel vs CodeQL — _h4xr · 2026-11-08
- leaf: a terminal Markdown previewer with GUI-like rendering and Mermaid support — 4310sy · 2026-11-08