GPT-6 Astra solves 98% of brand-new theoretical physics problems, catching experts' own calculation errors
新智元 · wechat · 2026-10-18
TheoPhysEval, built by physicists from ICTS, Harvard, UT Austin, Tokyo U and Google DeepMind with 50 unpublished theoretical physics problems, saw GPT-6 Astra score 88% initially and 98% after revision—because Astra insisted the human answer keys were wrong, and experts confirmed their own subtle errors in several problems. Claude Opus 5.5 scored 92%; 49/50 problems solved.
Even offline, Astra and Opus 5.5 hit 92%, with forced high-precision numeric outputs blocking guessing; per-problem API cost was under $3. An MIT study found frontier AI's solve rate on theoretical science problems jumped from 2% to 35% in two years (17.5x, still accelerating). All models failed one inverse-problem task, underscoring that human intuition still steers research.
Related event: GPT-6 Astra Solves 98% of Unpublished Theoretical Physics Problems(2 posts)→
More from Models
- LLMs used to be wrong, now they're bullshit, says Konrad Kording — KordingLab · 2026-10-19
- Pi harness nearly matches DeepSeek's own at 42.2 vs 42.5 on Bug Hunt Bench, at half the cost — PawelHuryn · 2026-10-19
- Krea2 uncensoring tools vanish: repo down, dev banned, refusal-reduction LoRAs removed — softlarch · 2026-10-19
- 'I miss 5.5': users say new models are better but lost their personality — Dapper_Ambition5963 · 2026-10-19
- Anthropic says dots will keep working in the EU and launch there later — Dimillian · 2026-10-18
- Gemini Fails to Identify the Prime Minister in Latest Capability Slip — LemonTeaCake007 · 2026-10-18