Frontier Math Problems Solved by Custom Models, Not General LLMs

ZeeshanZiaML · x · 2026-08-03

ML expert Zeeshan Zia shared insights from talks by math Prof. Sergei Gukov (Caltech) and Lijie Chen (OpenAI), clarifying misconceptions about AI solving advanced math problems.

He noted a misleading narrative on Twitter: the assumption that recent breakthroughs in long-standing math problems are achieved by general-purpose LLMs with harnesses like Lean. In reality, truly deep problems are being solved by highly custom models and harnesses (usually not LLMs) that build upon decades of manual progress on the structure of individual problems.

Problems falling to LLMs with general-purpose harnesses typically require shallow solutions but benefit from the LLM's broad familiarity across diverse math fields—essentially functioning as advanced RAG (RAG++), rather than demonstrating genuine deep reasoning capabilities.

Original post →

Frontier Math Problems Solved by Custom Models, Not General LLMs

More from AGI Musings