ICML 2025 Position Paper by 16 Researchers: Stop Treating AGI as AI's North-Star Goal
haydenfield · x · 2026-09-15
A discussion amplified by Margaret Mitchell highlights researchers questioning whether superintelligence should be pursued as a goal, with reporter Hayden Field noting a growing chorus of AI researchers warning: "Please don't build superintelligence soon. We are not ready. You need to slow down."
The centerpiece is an ICML 2025-accepted position paper, Stop treating 'AGI' as the north-star goal of AI research, by Borhane Blili-Hamelin, Margaret Mitchell, and 14 other authors. It argues AGI discourse aggravates six traps in goal setting:
- Illusion of Consensus
- Supercharging Bad Science
- Presuming Value-Neutrality
- Goal Lottery
- Generality Debt
- Normalized Exclusion
The authors call on the AI research community to (1) prioritize specificity in engineering and societal goals, (2) center pluralism about multiple worthwhile approaches to multiple valuable goals, and (3) foster innovation through greater inclusion of disciplines and communities.
More from AGI Musings
- Should AI models pass NIST-style safety checks before shipping? Industry voices say it's simple — aronchick · 2026-09-15
- Physicist Sabine Hossenfelder on arXiv and AI papers: flag them, don't ban them — skdh · 2026-09-15
- Common AI forecasts ignore algorithmic breakthroughs, argues one skeptical poster — shauseth · 2026-09-15
- Yacine: AI can't get much smarter — only faster from here — yacineMTB · 2026-09-15
- "Name One Thing AI Actually Made Better": Dev's Viral Skeptical Rant Beyond Coding — sebpaquet · 2026-09-15
- Analysis: Anthropic pacing alone buys one month; coordinated US slowdown cuts risk to 21% — soleio · 2026-09-15