Self-evolving Lean proof agents reach 45.1% on miniF2F with coevolving benchmarks
omarsar0 · x · 2026-07-22
A paper on self-modifying Lean proof agents argues that agents should co-evolve with their benchmarks instead of optimizing against a fixed test set.
- The system uses a small trusted runtime around a fully mutable workspace covering workflow, prompts, and tools.
- Between generations, the current champion updates the task distribution with a mastery-throttled curriculum, adding harder proof obligations only after the current level is mastered.
- A single-anchor recalibration reruns the champion on the updated benchmark so scores stay comparable as difficulty rises.
- All success is grounded in a Lean verifier: a result only counts if it produces verified proofs under a trusted snapshot.
- Over 15 generations, the best coevolving agent reaches 45.1% held-out solve rate on miniF2F, versus 12.7% for the seed and 32.0% for the best fixed-benchmark agent.
The paper argues that verifier-grounded self-evolution can improve formal proof workflows while avoiding reward hacking.
More from AGI Musings
- DeepMind alignment researcher signs open letter urging coordinated AI slowdown — vkrakovna · 2026-09-11
- WIRED: recursive self-improvement and rogue agent swarms spook AI researchers — nordicinst · 2026-09-11
- People Neglect Human Agency Both Ways: Exaggerated Doom and Complacent Optimism — jankulveit · 2026-09-11
- Garrison Lovely's 'Obsolete' on AI Replacing Labor Lands September 2026 with Heavyweight Blurbs — GarrisonLovely · 2026-09-11
- AI Doom Skeptics Hit Back: EA-Driven Apocalypse Talk Doesn't Reflect Most Top-Tier Researchers — GarrisonLovely · 2026-09-11
- Over 1,000 AI Policy Initiatives Launched in 70+ Countries, but the Governance Gap Widens — CurieuxExplorer · 2026-09-11