Gary Marcus probes math AI claims: how many solutions were tried and passed through Lean?
GaryMarcus · x · 2026-10-07
Gary Marcus presses for details behind a math AI system's headline results: how many candidate solutions were tried and passed through Lean verification, and how does the system actually work? Responding, altryne explains the Lean proofs ran as a separate process to "verify" outputs of the non-Lean agentic loop. Marcus's question highlights that search scale and pass rates remain undisclosed, making the claims hard to evaluate externally.
More from Research
- DFA transfers circuits from small to large models for cheaper interpretability, best on Llama-3 1B→3B — boknilev · 2026-10-07
- COLM paper: AI assistance reduces persistence and hurts independent performance — sethlazar · 2026-10-07
- Researcher predicts AI will solve every Millennium Problem and absorb all of mathematics — basedjensen · 2026-10-07
- Rutgers math chair on OpenAI's quasi-RH proof: a human would get an instant Fields Medal — burny_tech · 2026-10-07
- Soheil Feizi to speak on the structure of reasoning in LLMs at Simons Institute workshop — SharonYixuanLi · 2026-10-07
- DiVeR preprint trains VLA verifiers only on decision-critical states for test-time scaling — SharonYixuanLi · 2026-10-07