Anthropic's Agent4Sci: Frontier Agents Beat SOTA on 20 Scientific ML Benchmarks, but Only via Engineering Tricks
Anthropic · hf · 2026-10-22
Anthropic's Fellows Program introduces Agent4Sci, a long-horizon testbed measuring how far frontier coding agents can improve published-best models on scientific ML benchmarks under in-the-wild conditions.
- Spans 20 benchmarks across 11 domains; instead of a leaderboard, it asks what progress agents make and where they fall short
- Today's frontier agents advance the published best on every benchmark, with Claude Opus reaching 10.4% mean relative improvement and no run flagged for academic misconduct
- Key finding: the gains come almost entirely from a bag of ML-engineering tricks rather than research-level insight
- Trajectory analysis surfaces three recurring limitations, and targeted skills significantly boost performance
More from coding & agent
- When AI automation saves less than it costs: the hidden maintenance tax of agents — djgoel123 · 2026-10-22
- PHP ships emergency fixes for two unauthenticated RCE bugs after embargo break — kingslyj · 2026-10-22
- Flying a drone with Codex: 12–20s per decision is the bottleneck — kfountou · 2026-10-22
- One week of vibe coding, 30,708 players in month one: Dust 2 DM game's Spawn stats — TAbrodi · 2026-10-22
- Security patch ships with 1,858× speedup: 3,000-scene queries drop from 2s to 1.13ms — DanielLockyer · 2026-10-22
- Codiff v1.18 ships faster startup, prettier markdown diffs and a new app icon — cnakazawa · 2026-10-22