h1 Paper Shows Bitter Lesson in Automated Theorem Proving: Simple Harness Hits SOTA, Solves 4 Erdős Problems
burny_tech · x · 2026-11-03
The KempeLab team's h1 paper is finally on arXiv after a 30-day hold, titled 'A Taste of the Bitter Lesson in Automated Theorem Proving.'
Origin: Inspired by capabilities unlocked at test time (think step by step → CoT → +RL), the authors tested whether test-time asymmetric self-play could unlock hard problems, using Lean since posing a statement fully defines a verifiable task.
Finding: A 'challenger' analyzing 'prover' agent traces to induce specializations/generalizations kept washing out gains whenever they carefully adjusted baselines. The winner was h1 — a minimal harness giving the model an efficient Lean 4 interface, kept working via verifier feedback and self-reflection, with no imposed decomposition or search structure.
Result: This simple baseline is SOTA and solved 4 extra Erdős problems. The paper argues formal verification must scale to keep up with frontier models, and that workflow gains are often confounded with stronger models or more compute.
More from coding & agent
- Dev shares 6-step privacy-safe analytics setup for a WhatsApp agent using SHA256-hashed IDs — petrusenko_max · 2026-11-03
- Dev embeds in-context feedback to kickoff background agents fixing his app in a couple clicks — SubtleGradient · 2026-11-03
- LLM coding agents write from-scratch LP solvers averaging ~10x faster than general-purpose ones — sh_reya · 2026-11-03
- Three.js open sea lighting demo built with Claude Opus 5.5 — techartist_ · 2026-11-03
- Agent-built data pipeline flags law firms whose clients later hit SEC/DOJ trouble — doodlestein · 2026-11-03
- Film director vibe-codes a 30-minute horror game entirely with Claude — ArkhamDreamerZero · 2026-11-03