First Proof batch 3: mathematicians grade GPT-6, Opus 5.5 and Kimi K3 on research-level proofs
littmath · x · 2026-10-15
The First Proof project — run by top mathematicians including Abouzaid, Srivastava, Ward and Williams — has released results for its third batch of ten research-level math problems, testing whether current AI systems can correctly solve problems drawn from the contributors' own research. Solutions were graded on mathematical correctness, exposition quality, and attribution/citations.
From the results heatmap (correctness / exposition / attribution scored separately):
- GPT-6 Astra and the multimodel ProofCouncil lead, with most scores above 0.6
- Siege (Opus 5.5) failed to submit on several problems
- MerLean (GPT-6) is polarized, with badly negative attribution scores on some problems
- Humanize (Kimi K3) trails, scoring below -0.6 on many items
Full human solutions, AI solutions, and referee logs are published alongside the report — a rare expert-led benchmark built on genuine research problems.
More from Models
- SWE-Bench Multimodal Hits Vals: Every Claude Model Outperforms OpenAI's — dejavucoder · 2026-10-15
- Scale CEO spotlights Muse Wonder: untrained for 3D yet beats Gemini 4 in physics visualization tests — alexandr_wang · 2026-10-15
- Cohere tops WMT26 translation benchmark: small agentic model scores 84.36, beating Mistral and DeepL — cohere · 2026-10-15
- Anthropic has never launched a Sonnet/Opus-tier model on a Wednesday — ChrisGPT · 2026-10-15
- 8k context beats 480k: OpenAI Decisions API cuts costs 84% at 98.25% F1 — OpenAIDevs · 2026-10-15
- Gemini 4 Argon hit with HTTP 524 timeouts outside Antigravity; Google payment also glitching — cedric_chee · 2026-10-15