TheoPhysEval: GPT-6-Astra agent hits 98% on 50 unpublished research-level physics problems
rohanpaul_ai · x · 2026-10-25
Google DeepMind and 30+ researchers across top universities released TheoPhysEval on arXiv: 50 unpublished research-level theoretical physics problems with verifiable answers, each solvable by a world expert in about a day without AI.
- Off-the-shelf agents were tested autonomously with realistic tools: literature search plus numerical/symbolic scripting
- The best agent, GPT-6-Astra harnessed in Codex, achieved near-perfect 98% accuracy at under USD 3 per problem
- Denying web search and background papers caused only small accuracy drops, suggesting significant memorization of the literature
More from Models
- Thinking Machines pivots from data to models and immediately hits Pareto SOTA — nirmal_dist · 2026-10-25
- Kimi K3.1 Pro vs Claude Opus 5.5: 13 agentic coding tasks for 55% of the bill — Prompt Engineering · 2026-10-25
- Nanbeige Releases New Nanbeige4.3-33B Model — DotaMate · 2026-10-25
- Interconnects podcast: Kimi K3.1's mixed reaction, open-weight cyber risks, expert divides — natolambert · 2026-10-25
- vibesbench: an informal benchmark where models opine on 90s alt-rock angst — firasd · 2026-10-25
- RIP unlimited Claude: Pro user's limits maxed out as Anthropic tightens quotas — prasenx · 2026-10-25