Astra Loses to 1300-Elo Chess Bot, Then Wins with Ultra Mode and Visualization Tool

On 09-06, MikePFrank posted a multi-part experiment letting AI model Astra play chess directly against chess.com's Nelson bot (ELO 1300, intermediate). Unaided, Astra blundered repeatedly and resigned; even at Extra High reasoning it still lost. But switched to Ultra mode and armed with a self-built single-step board visualization tool, Astra dominated Nelson and won. The series exposes frontier models' weakness in long-horizon strategy while showing how coding and agent abilities can compensate.

Confirmed

• At High reasoning, unaided, Astra kept pace early but made multiple obvious blunders and resigned (m1, m6, m7).
• Retried at Extra High: slightly better, still lost to Nelson (m3, m4, m7).
• No external engine cheating (m1).
• After the game Astra wrote a browser-based game replay app on the spot; the author also had it add a 'single-step board visualization' capability to avoid low-level mistakes (m3, m4).
• Turning point: Astra (Ultra) with the visualization tool completely dominated Nelson, whose position collapsed, and won; a full replay link was shared (m5, m6).
• The author noted LLMs have always been weak at out-of-the-box chess, but modern models can write complete chess engines—letting Astra try compensating via code (m2).

Why it matters

• The series vividly shows frontier models' shortcomings in long-horizon strategic reasoning: even at Extra High, they lose to a 1300-Elo intermediate bot.
• But the ending isn't just a flop: Ultra mode plus self-installed tooling reversed the result, showing an alternative path—models can write engines and visualization tools on the fly, using coding and agent abilities to offset native reasoning limits, a classic 'agentic AI' play.
• It also shows that the same model's capability can vary greatly across reasoning tiers and tool setups; a single game is insufficient to judge a model.

2026-09-06 ~ 2026-09-06 · 7 related posts

Full story(5 episodes)→

• Episode 1: Astra Loses to 1300-Elo Chess Bot, Then Wins with Ultra Mode and Visualization Tool(2026-09-06, 7 posts)
• Episode 2: GPT-6 Astra beats chess engine with raw model output, 41-move mate(2026-09-06, 3 posts)
• Episode 3: OpenAI's Astra Loses to 1800-Elo Chess Bot After Trapped Bishop Blunder(2026-09-06, 7 posts)
• Episode 4: Astra Scores Over 1800 Elo in Chess Without Engines(2026-09-07, 5 posts)
• Episode 5: AI Agent Astra Revamps Engine, Checkmates 1800-Elo Chess Bot in 31 Moves(2026-09-08, 6 posts)

• LLMs Play Chess Poorly, So This One Built Its Own Chess Engine to Fight Back — MikePFrank · 2026-09-06
• [source] Astra blunders its way to resignation in chess test, rerun at Extra High reasoning — MikePFrank · 2026-09-06
• Astra agent blunders its way to chess resignation, gets a rematch at Extra High — MikePFrank · 2026-09-06
• AI model loses chess match against chess.com's 1300 ELO Nelson bot — MikePFrank · 2026-09-06
• Astra Ultra with visualizer now dominates Nelson as gap widens — MikePFrank · 2026-09-06
• [source] Astra Ultra with move visualizer trounces Nelson chess bot (ELO 1300) — MikePFrank · 2026-09-06

1 near-duplicate retellings: MikePFrank