AI Agents Fail at Open-Ended Research: Automation Remains Out of Reach
sudoraohacker · x · 2026-08-01
Countering optimistic visions of Recursive Self-Improvement (RSI) and automated research, researchers highlight severe limitations in current AI agents based on a recent paper and 'shadow evaluations'.
Core Experiment & Findings:
- Agents were given 6 days and thousands of dollars in API credits to replicate two unpublished papers. The original authors unambiguously rejected the AI-generated outputs.
- While fluent at engineering tasks, agents struggled significantly with open-ended research requiring high-level judgment and tacit knowledge.
Why Cutting-Edge Research is Hard to Automate:
- High Iterative Complexity: Producing a top-tier paper requires numerous iteration cycles across ideation, design, execution, writing, and review—far more complex than typical white-collar labor.
- Unpredictable Failure Modes: Delegating these processes to AI introduces dozens of emergent errors. Human experts use tacit knowledge to refine work, whereas AI models are just as likely to introduce new errors during iteration.
- Underestimation of Difficulty: Those conjecturing about the automation of scientific research severely underestimate the actual threshold for producing novel, validated scientific work.
More from coding & agent
- Harness open-sources desktop and mobile apps for orchestrating agent fleets across frameworks and machines — dee_hw · 2026-10-16
- Open-Source Google Analytics MCP Server Lets LLMs Query GA Data via Natural Language — modelcontextprotocol · 2026-10-16
- LLM ideation experiment: just ask for non-obvious ideas — and Opus beats smaller models by a wide margin — arjunrajlab · 2026-10-16
- a16z podcast: AI cut a 6-month feature to 10 days and widened engineer gaps 10x — a16z · 2026-10-16
- "Hold On, My Agents Are Freezing My Screen": The Daily Grind of the Agent Era — every · 2026-10-16
- Dev builds MCP feeding Claude 3,000 real launch videos to fix samey AI output — Designer_Landscape25 · 2026-10-16