Apodex launches TRACES, a benchmark grading AI's reasoning trajectories on unsolved science problems
rohanpaul_ai · x · 2026-09-04
- Apodex released TRACES, a benchmark built for hard scientific problems where the correct answer may not exist yet — addressing the gap that most benchmarks assume an answer exists.
- How it works: problems become executable environments where a solver can use data and tools, receive feedback, and revise its approach, leaving a full trajectory. Trajectories are scored across six capabilities: Tools, Repair, Alternatives, Coherence, Evidence, and Scope, with a hidden verifier separately evaluating the outcome — so failures can be traced to wrong tool choice or failed recovery.
- The technical report, led by chief scientist @profshengwang, involved 10 STEM PhDs over two months surveying 561 industries to curate 423 real high-stakes problems spanning biomedicine, clinical translation, and frontier-model engineering. Both problem and solver submissions are open.
More from Research
- Sabine Hossenfelder: Science Must Drop the Requirement That Humans Can Understand It — AI Is the Way Forward — coherence · 2026-10-21
- A real larva's connectome — 2,675 cells, 14,066 connections — now swims in a 3D simulation — eptwts · 2026-10-21
- Stanford's T2-z lets robots probe hidden physics with a poke, lifting success from 34% to 73% — stanfordnlp · 2026-10-21
- Fei-Fei Li team unveils FACT, a force-aware foundation model for robot dexterity — stanfordnlp · 2026-10-21
- Upcoming paper claims big models can handle latency-critical real-time robotics — ManlingLi_ · 2026-10-21
- NVIDIA open-sources Flora, a robotics data engine 125x cheaper than GPT-6 Astra — DrJimFan · 2026-10-21