Anthropic's Agent4Sci: Claude Opus beats published SOTA on all 20 science ML benchmarks
dair_ai · x · 2026-10-22
Anthropic published Agent4Sci, a paper on AI agents doing ML-for-science research. It gives frontier coding agents 20 scientific ML benchmarks across 11 domains and asks them to beat the published best model — they beat it on all 20, with no runs flagged for misconduct.
- Claude Opus improves on the published best by 10.4% on average; Codex by 4.4%.
- The gains come almost entirely from standard ML-engineering tricks; trajectories show agents misjudging novelty, leaving compute unused, and settling into small local sweeps.
- Adding three lightweight skills — research ideation, experiment management, and scheduled checklists — lifts Opus from 10.4% to 20.1% and Codex from 4.4% to 8.1%, with more varied, domain-relevant changes.
Key takeaway: with the same models, the research scaffolding determines the ceiling of agent scientific capability — highly relevant for anyone building AI-scientist systems.
More from coding & agent
- Pi Coding Agent Gets a Full Rust Port via Claude, Under 90MB Memory — jasonkneen · 2026-10-22
- Securing coding agents with Docker Sandboxes and edge LLM inference at ATO — HowDevelop · 2026-10-22
- Dev Builds MCP Memory That Keeps Project Notes Separate Instead of One Summary — Jolly-Scientist-2484 · 2026-10-22
- Free book 'LangGraph in Small Pieces' teaches agents from blocks to production — truckkicker · 2026-10-22
- What Everyone Is Getting Wrong About Context, per a Founder's New Article — shensi · 2026-10-22
- Cloudflare's AI security-audit skill tops GitHub with 38.5k stars — alex_verem · 2026-10-22