StarSkirmish Bench: LLMs write StarCraft Brood War bots in one hour, GPT-6 and Opus 5.5 lead
philipvollet · x · 2026-09-27
StarSkirmish Bench gives each LLM one hour to write a Protoss bot in C++ for StarCraft: Brood War (BWAPI 4.4.0 on OpenBW, three ladder maps), scored by win rate against human-written and demo bots (top human bot Stardust = 100, weakest demo = 0).
Key points:
- Tests long-horizon reasoning and agentic coding: LLMs get tools to compile, run practice batches, and read game transcripts with build timings and fight details.
- GPT-6 Astra and Claude Opus 5.5 are functionally tied at the top; with GPT-6 Sol they clearly lead all other models. On a score-vs-cost log plot, GPT-6 Sol offers uniquely good value.
- Games reaching the 60-minute cap (86,400 frames) are decided by kills + razings as on the BASIL ladder.
Related event: StarSkirmish Launches: LLMs Write StarCraft Brood War Bots to Battle(2 posts)→
More from coding & agent
- Explosion launches Ellf directory: 52 NLP templates plus multi-agent annotation workflow — _inesmontani · 2026-11-11
- Meme: 'Stochastic AI Agent Architecture' Is Just a Markov Chain in Disguise — PrasoonPratham · 2026-11-11
- OpenCode adds Gemini 4 Carbon and Claude Fable 5.2 support — dean_rie · 2026-11-11
- OpenClaw v2026.11.3 patch fixes cert verification and credential exposure across 56 PRs — dean_rie · 2026-11-11
- Fable 5.2 captured 1 in 4 Claude replies on day one, then growth stalled — dean_rie · 2026-11-11
- QA engineer proposes MCP agent to automate ad postback checks before every app release — Soobhee_Chukwutosin · 2026-11-11