Formalizing Julia in Lean with AI agents uncovers ~400 bugs in Base
ericjang11 · x · 2026-10-30
Miles Cranmer reports on the Julia Discourse that formalizing software in Lean 4 — with AI agents writing the proofs — is a powerful bug-catching technique inspired by recent security research.
- Method: each Julia function gets a Lean model; docstring promises become theorems the agents try to prove. Failed proofs usually yield counterexamples that turn out to be real bugs.
- Results: his own heavily tested, 100%-human-reviewed DynamicExpressions.jl yielded tons of hidden bugs; formalizing Julia's full source code has so far found 400 bugs in Base.
- Takeaway: Base isn't unusually buggy — Lean formalization is just extremely effective, and strong AI math performance makes it scalable in a way that was previously impractical.
Eric Jang amplified the post, quoting the comment section's "clanker based correctness."
More from coding & agent
- YC F26's Composal sends agent swarms to test your product like real users — ycombinator · 2026-10-30
- KreaMossMix ComfyUI workflow chains MOSS-Image and Krea 2 for four image variants per prompt — Plane-Implement1044 · 2026-10-30
- Dev builds fail-closed MCP tool that refuses to PASS unverified US businesses — cstill_dev · 2026-10-30
- Claude Code v2.1.314 adds write-blocking outside workdirs, idle compaction, gateway fixes — ashwin-ant · 2026-10-30
- ComfyArt: open-source Lovart-style AI design canvas running fully on your own GPU — Far-Driver-2904 · 2026-10-30
- Particle-1, a $0.25/$1 codegen model with 1M context, goes live on agent harness Capy — garrytan · 2026-10-30