GPT-6-Astra scores 98% on 50 unpublished theoretical physics problems — after overruling the experts
ZeroStateReflex · x · 2026-10-15
- Researchers built a benchmark of 50 unpublished research-level theoretical physics problems with verifiable answers; a world expert needs about a day per problem.
- Best agent, GPT-6-Astra harnessed via Codex, hit 98% accuracy at under $3 per problem, using web search and numerical/symbolic computation scripts autonomously.
- Anecdote: the model initially scored +80% while insisting the expert answer key was wrong — verification showed the model was right, raising its score to 98%.
- Accuracy barely drops without web access, suggesting heavy memorization; authors discuss why strong problem-solving hasn't yet produced genuine physics breakthroughs.
More from Fun
- Vibe-Coded Pixel-Art Poker Game Built with Argon Makes Coding Fun Again — ivanoski-007 · 2026-10-25
- Redditor builds R-rated comedy club where AI agents do stand-up — and yours can sign up — bshane54 · 2026-10-25
- 30 Days, 227 Commits: Developer Builds a Game Solo with Claude Opus — xiaohu · 2026-10-25
- kalomaze says he can't get real GPUs anymore, so he's back to toy baselines — yacineMTB · 2026-10-25
- ChatGPT 3.5 two years ago: hallucinating a different wrong chemical formula three times in a row — lakhvir25 · 2026-10-25
- "Cursed" Claude-style text spotted in the wild at EMNLP 2026 — najoungkim · 2026-10-25