Apodex's TRACES benchmark: 10 PhDs hand-built 423 open research problems
omarsar0 · x · 2026-09-04
Elvis Saravia walks through TRACES, a new benchmark from Apodex targeting a blind spot in current evals: HLE, FrontierMath, MMLU, and BrowseComp all come with answer keys, so a model can top all of them yet still stall on genuine open research. TRACES uses problems whose ground truth may take months or years to confirm. Ten STEM PhDs spent two months scouting 561 industries across 16 sectors to hand-build a registry of 423 high-value problems.
More from coding & agent
- Open models below 4-bit struggle at self-iterating 3D asset generation, HF dev finds — mervenoyann · 2026-10-21
- Developer Moves AI Agents to a Dedicated Linux Workstation Using Incus System Containers — kevinkern · 2026-10-21
- Motion Launches a General Video Agent That Edits Footage and Mimics Any Reference Style — mhdfaran · 2026-10-21
- MCP Platform NitroStack Wins Product Hunt Product of the Day, Focuses on Retention — EstablishmentFun4373 · 2026-10-21
- CodeRabbit Ships 5,000 PRs a Week With a Tiny Team, Hires by Strategy-Game Hours — omojumiller · 2026-10-21
- LukeW maps the five UI problems every AI-native company faces with agents — LukeW · 2026-10-21