AI Agents Wrote Two Papers in 6 Days for $3K, Both Rejected for Lack of Judgment
rohanpaul_ai · x · 2026-08-01
An experiment allowed AI agents 6 days and a $3,000 budget to conduct open-ended research autonomously, resulting in two papers that were both rejected by human experts.
- Strong Execution, Poor Judgment: The agents (primarily Claude Opus 4.8) successfully ran hundreds of experiments, debugged crashed GPUs, and compiled LaTeX without human intervention. They also avoided reward hacking, honestly downgrading claims when facing negative results.
- Core Flaw: When confronted with negative automated reviews, the agents failed to redesign their experiments. Instead, they merely narrowed their claims and added more caveats.
- Underutilized Resources: Both runs ended with over half of the $3K budget unspent, indicating the agents failed to recognize they were short on ideas, not money.
Related event: AI Agent Autonomous Research Fails: Papers Rejected(2 posts)→
More from coding & agent
- Coding agent runs are getting long — one dev suggests breathing, not parallel sessions — teropa · 2026-10-16
- Dev builds MCP layer so Claude Code can work on servers without ever holding SSH keys — NETMESAFE · 2026-10-16
- As coding agents run longer autonomously, one dev's tip: don't parallel-session, just breathe — teropa · 2026-10-16
- 'Refusing to code with AI in 2026 is like refusing Google in 2005' — prasenx · 2026-10-16
- Claude Code adds idle compaction to save 1h prompt cache on idle sessions — EricBuess · 2026-10-16
- Handling agents that crash mid-task: the hard part is the unknown outcome across the network — Responsible-Dare5139 · 2026-10-16