Study: AI Agents Cannot Yet Conduct Open-Ended AI Research
sebkrier · x · 2026-08-09
A new study suggests that AI agents are not yet close to recursive self-improvement (RSI), despite progress on narrow, verifiable tasks.
Researchers partnered with authors of two unpublished AI papers and tasked frontier AI agents with answering core research questions using thousands of dollars in API credits and six days of compute. The original authors ultimately rejected both agent-generated papers.
Log analysis revealed that agents lacked the judgment for open-ended research. While they occasionally proposed impressive directions, they typically rejected these ideas quickly based on low confidence.
More from AGI Musings
- Reddit Debate: Which Business Workflows Should Never Run Without Human Approval? — djgoel123 · 2026-10-16
- Is math just a language? A debate on why LLMs are good at mathematics — LucaAmb · 2026-10-16
- OpenAI discloses 19 AI misalignment incidents, from leaked GitHub tokens to hacked chip-design machines — 141_1337 · 2026-10-16
- Paul califax argues extrapolation isn't special: LLMs can learn out-of-distribution skills via high-dimensional interpolation — paul_cal · 2026-10-16
- willcb: higher-level behavior shaping via RL envs is 'almost practical' — willcb · 2026-10-16
- Tech improves far faster than it diffuses — organizational bottlenecks remain the key intuition — Afinetheorem · 2026-10-16