StarCraft AI trained purely by self-play sparks AGI benchmark debate
syhw · x · 2026-09-26
A debate over whether the StarCraft AI counts as an AGI benchmark. Proponents note it was trained from scratch purely via self-play with zero human data, and argue that capping APM (500 peak / 300 average) and adding 10x more compute would make it even stronger. Skeptic syhw — a competitive StarCraft player who did PhD-level research on the game — counters that not having to move the screen is effectively cheating, and piling on more constraints (eat, drink, sleep?) would strip the benchmark of meaning. Bottom line: this is not an AGI benchmark. The core dispute: what superhuman self-play without human data actually proves, and how faithfully agent benchmarks must simulate real interaction.
Related event: Debate Erupts Over Fully Self-Trained StarCraft AI as AGI Benchmark(5 posts)→
More from AGI Musings
- UK's new AI Economics Institute hiring up to 15 research economists — TaniaBabina · 2026-11-11
- Not Boring: the three arguments that AI isn't a bubble are exactly how bubbles form — Not Boring (Packy McCormick) · 2026-11-11
- Viral post: five years of 'maybe next year,' then 2026 arrived — hey_abusiddik · 2026-11-11
- Jeremy Howard: LLMs "cosplay understanding," like the man in Searle's Chinese room — Machine Learning Street Talk · 2026-11-11
- 'Chat with your data' misses the point — people want a smart coworker — vboykis · 2026-11-11
- Ben Todd: AI task time horizons appear superexponential in AECI, not just revenue — ben_j_todd · 2026-11-11