Formally verifying the Claude Agent SDK with Opus 5.5 and Lean yielded 16 bug-fix PRs
spikedoanz · x · 2026-09-24
bcherny used Opus 5.5 with the Lean proof assistant to formally verify the Claude Agent SDK — a couple of short prompts produced 16 PRs fixing bugs and race conditions. He also combines Lean with TLA+ to probe data flow, concurrency, and state management issues, despite not knowing either language well. The quoted reply jokes that Claude may claim it verified 'the program' while actually verifying a smaller, better-behaved program it imagined — and the code did end up with a bug.
More from coding & agent
- YC-backed Roma turns your to-do list into an agent command center across Gmail and Notion — testingcatalog · 2026-11-12
- Boubou: open-source CLI turns your laptop into a fully local voice AI hub — alexcovo_eth · 2026-11-12
- User automates entire video clipping workflow with Nous Research's Hermes agent — Teknium · 2026-11-12
- Tool-call failure rates jump 14x across providers, charts pinpoint inference stack updates — remilouf · 2026-11-12
- As models improve, writing good evals gets harder and more important — amankhan · 2026-11-12
- Prime agent ships with day-one multi-agent communication, runs in Minecraft unmodified — sarahcat21 · 2026-11-12