CritPt v2 hard subset: GPT-6 Astra leads at 72%, top open-weights model only 45%
ArtificialAnlys · x · 2026-10-28
Artificial Analysis explains how it scores CritPt v2 in its Index: a private 40-question hard subset of 70 research-level physics problems, minus 30 that seven frontier models solved in all five attempts at launch. Models run in an offline Python sandbox via its agentic harness Stirrup.
GPT-6 Astra leads at 72% (high effort), GPT-6.1 Sol follows at 69% (max effort), and overall Index leader Claude Opus 5.5 (max) scores 61%. The best open-weights model, Kimi K3.1 Pro, trails at 45%.
More from Models
- Debate: Claude isn't 'lobotomized' — inference stack updates may shift old-model outputs — JasonBotterill · 2026-10-28
- AI labs near IPO, yet LLMs still produce 'astonishingly terrible prose' by default — GabGarrett · 2026-10-28
- Rebuttal: Kimi 3.1 Nears Fable on Frontier Code 2, Contradicting 'China Models Crushed' Claim — bookwormengr · 2026-10-28
- Open model Flint hits #2 trending on Hugging Face, team outlines next steps — josh_tobin_ · 2026-10-28
- Artificial Analysis Intelligence Index v5 widens open-weight gap from 3 to 12 points — ResearchCrafty1804 · 2026-10-28
- YacineMTB rails: even the best LLMs are 'complete dogshit' and he's tired of it — yacineMTB · 2026-10-28