GPT-6 Astra scores 98% on expert-crafted theoretical physics benchmark, solving day-long problems for under $3
alex_verem · x · 2026-10-16
Physicists from Harvard, Duke, University of Tokyo and elsewhere built TheoPhysEval: 50 unpublished research-level theoretical physics problems with verifiable answers, each taking a world expert about a day.
- GPT-6-Astra (in Codex), with web search and code execution, solved 49 problems at 98% final accuracy, under $3 per problem
- Claude Opus 5.5 solved 46, Gemini 3.8 Flash 34, and best open-weight DeepSeek-V4.1-Flash 27
- With search disabled, Astra and Opus still solved 46 each — authors suggest substantial memorization of the physics literature
- Astra repeatedly insisted the experts' own results were wrong; the physicists rechecked and Astra was right
- Authors call it a milestone: the calculating part of physics research is now cheap, but choosing which problems matter remains a human job — agents asked to invent problems gravitate to details humans find uninteresting
More from Models
- Zuckerberg: Meta's Muse Spark Open weights out today, runs on Mac Studios — AIatMeta · 2026-10-16
- NVIDIA releases open Nemotron 3.5 Super VL: 120B params, 12B active, multimodal — yoavartzi · 2026-10-16
- Meta's Muse Spark Open: 816B/A42B, 1M context, top US open-weight model — alexandr_wang · 2026-10-16
- Microsoft unveils MAI-Base-2: 2.8T-param MoE, 4-12x more pre-training FLOP-efficient than DeepSeek — ChengleiSi · 2026-10-16
- Pro Users Say They Can't Access Gemini 4 Argon, Call Antigravity 'Nearly Unusable' — realmeetjames · 2026-10-16
- Tiny classifier face-off: Vega runs on CPU, phone, browser at 75ms on M5 — alexcovo_eth · 2026-10-16