PhysClosureBench: Opus 5-5 Tops Atomic Score but Closes Research in Only 32.7%
Shanghai-AI-Laboratory · hf · 2026-10-19
Shanghai AI Lab, SJTU, HKUST(GZ) and NTU introduce PhysClosureBench: 50 expert-verified, goal-specified research tasks in strong-field, attosecond and neighboring physics where agents organize the investigation and validate evidence themselves. A 20-criterion Atomic Score measures local scientific accomplishment; Research Closure (RC0–RC4) measures whether the question is fully resolved with indispensable evidence and applicability limits satisfied. Across seven frontier LLMs, Claude Opus 5-5 achieved the highest mean Atomic Score (6.29/10) but reached RC4 in only 32.7% of attempts; benchmark-wide, 36.2% of attempts scoring ≥8/10 still fell short of RC4 — agents produce strong local results but fail to fully close research questions due to incomplete evidence and unresolved applicability limits.
More from AGI Musings
- Mollick invokes SAGE: MIT and IBM pioneered computerized management decisions decades ago — emollick · 2026-10-19
- Amazon, Delta and JPMorgan are blocking AI agents — a classic App Store moment? — Yuchenj_UW · 2026-10-19
- Thought experiment: if billion-dollar companies stopped open-weight releases for a year — ForsookComparison · 2026-10-19
- Counter-meme take: in an AI world, the smartest humans may matter more than ever — shakoistsLog · 2026-10-19
- Automation paradox: agriculture fell from 40% to 2% of GDP while output soared — zacharynado · 2026-10-19
- The Dark Ages as a template: why civilizational decline can seed a new order — floguo · 2026-10-19