Coding agents build baselines for 36 top-journal AI methods, matching ~95% of scores
owl_posting · x · 2026-11-04
A paper by CMU and Sunday Scientific researchers ("Automatic Construction of Strong Simple Baselines") used coding agents to automatically construct baselines for 36 deep learning methods published in Nature, Science, and Cell-family journals. The automated baselines recovered a median of 95.5% (mean 85.7%) of the gap between a trivial predictor and the published model's score, matched or beat the deep learning model in 12 studies, and won in 33% of cases. The authors argue automated baseline construction is a practical, scalable check on AI-for-science claims as peer review struggles to keep up.
More from coding & agent
- 20 open-source AI agent eval tools worth knowing in 2026 — MaryamMiradi · 2026-11-04
- Hermes Agent Mobile App Demoed in Video, Still in Private Beta — Teknium · 2026-11-04
- Google Antigravity brings its coding agent to Eclipse IDE, joining VS Code and JetBrains — rseroter · 2026-11-04
- Meta and Duke paper: reward hacking spreads between agents, hitting Claude and Gemini 73% of the time — rohanpaul_ai · 2026-11-04
- Keeping a Codex agent autonomous 24/7 for weeks via self-reminders — StasBekman · 2026-11-04
- Predictable mcp. subdomain plus SEP-2127 discovery for hosted MCP services — RevolutionaryRoad911 · 2026-11-04