Kenneth Stanley on why measuring open-endedness is so hard for AGI
mhmazur · x · 2026-10-26
Notes from Kenneth Stanley's ARC Prize conference talk: sustained invention and discovery are essential to AGI, but benchmarking them is fraught—a poorly chosen measure can steer research away from the discoveries we want.
- Open-endedness: processes that keep innovating indefinitely, like biological evolution and civilization; both accumulate "stepping stones" within a single ongoing run, unlike goal-directed AI search.
- Why it matters: strong test scores leave creative contribution unaccounted for; Stanley argues AGI and superintelligence require open-endedness.
- Two lessons: narrow objectives hide useful stepping stones; equal benchmark scores can mask different internal representations with very different future creative potential.
- Three measurement approaches (activity statistics and others) each work in scope but none suffices as a general frontier-ranking benchmark.
- The field traces back to artificial-life "open-ended evolution" research, entering mainstream AI discussions since the late 2010s.
More from AGI Musings
- Frontier LLMs negotiate contracts poorly and frequently renege for selfish gain, study finds — xuanalogue · 2026-10-26
- Study of 133 patent lawyers: AI augments veterans but causes cognitive offloading in novices — xiaohu · 2026-10-26
- OpenAI computer-use agents near superhuman speed, say the team building them — a16z · 2026-10-26
- Kai-Fu Lee publishes AI Native, a roadmap for building AI-native companies — kaifulee · 2026-10-26
- RCT: ChatGPT temporarily boosts consultants' data science skills, gains vanish without it — EricTopol · 2026-10-26
- Dev on AI games backlash: reject AI slop, not the tool itself — ezshine · 2026-10-26