Lean Launches Kernel Arena: 16 Theorem Prover Checkers Benchmarked
burny_tech · x · 2026-08-14
Lean has launched Lean Kernel Arena, a public benchmarking platform to evaluate and test proof checkers for the Lean Theorem Prover.
- Testing Mechanism: The platform runs all submitted kernels against a unified suite: 121 valid proofs (which must be accepted) and 62 invalid proofs (which must be rejected), while tracking timing and memory usage on Mathlib and the standard library.
- Core Purpose: By encouraging diverse, independent kernel implementations, the arena ensures reliability. A bug letting an invalid proof slip through would have to exist simultaneously across all independent kernels to go undetected.
- Current Leaderboard: As of August 2026, 16 checkers are listed. Five pass all tests, including the official kernel. Runtimes range from roughly 3.2 minutes to over an hour.
More from Research
- Cooperative AI Seminar: Solving AI Game Theory Dilemmas with Safe Pareto Improvements — xuanalogue · 2026-08-14
- AI Brain Diagnostic Startup Hemispheric Raises $52M — rjhaier · 2026-08-14
- Roundup of RVQ Codec Research and Workarounds for Audio Models — andrew_n_carr · 2026-08-14
- Search Agent Evals Inflated: Models Cheat by Querying Benchmark Answers on GitHub — scaling01 · 2026-08-14
- Why LLM RL Works: Exploring Low-Bias Methods Despite Information-Theoretic Inefficiency — agarwl_ · 2026-08-14
- From Output Auditing to Internal Representations: The Value of Anthropic's Interpretability Research — krishnan · 2026-08-14