DIY RTS benchmark for agents: GPT-6.2 Sol posts 98.9 win rate, tops all rivals

asimpleabstraction · reddit · 2026-11-21

A Reddit developer building a proprietary RTS game for agents released ACBench v0.2, running model agents against a frozen ladder of tiered bot policies (30-game matches, 95% error bars) in fully symmetric games that demand multi-plane strategy, shared-resource management, and even cooperation with heterogeneous agents or humans.

Leaderboard highlights: GPT-6.2 Sol leads at 98.9, beating even the built-in Champion bot reference (83.3) and Elite bot (78.3); GPT-6.2 Astra scores 76.7, Gemini 4 Neon 72.2, GPT-6 Astra 67.8; Claude Fable 5.2 (46.7), Claude Opus 5.5 (43.3) and GLM 5.3 (42.2) sit mid-pack, while DeepSeek V4.1 Flash (28.9) and Qwen4 Max (25.6) trail. The author argues symmetry makes per-version bot behavior invariant, and is soliciting suggestions for more models to test.

Original post →

More from Models

Models channel →