Meta FAIR's h1: simple agentic loop hits SOTA in formal theorem proving
meta-llama · hf · 2026-10-28
Meta FAIR (with EPFL, ENPC, NYU) presents h1, testing the bitter-lesson question in formal theorem proving: can a simple agentic loop match human-designed workflows?
- Approach: a minimal harness giving the model an efficient Lean 4 interface, driven only by verifier feedback and self-reflection—no decomposition or search structure.
- Results: SOTA on PutnamBench, FATE-X, and FormalConjectures100-Solved; settles 8 open problems in FormalConjectures; matches or beats specialized workflows like Goedel-Architect and AlphaProof Nexus at matched models, often with a single attempt (pass@1).
- Key finding: ablations and matched-model comparisons with Claude Code and AxProverBase show simplicity alone isn't enough—an efficient Lean interface largely determines cost and solve rate.
- Recommendation: evaluate frontier models with efficient execution feedback before prescribing workflows, at the risk of bitter lessons.
More from coding & agent
- Claude Catches North Korean Supply Chain Attack Disguised as an Auth Error — banteg · 2026-10-28
- Expo ships agent skills collection for AI coding tools, already 3.3k GitHub stars — expo · 2026-10-28
- Does a Google Play uploader MCP exist, and would you trust an agent to ship to production? — Extender7777 · 2026-10-28
- Lucas Meijer: writing and evaling skills forces you to say what you care about — lucasmeijer · 2026-10-28
- Merge CEO Shensi Ding stays hands-on with Codex: execs set the AI fluency ceiling — OpenAI · 2026-10-28
- Detection Isn't Blocking: How to Actually Verify Your AI Agent's Security Guard — shoumikgoswami · 2026-10-28