Astra Scores Over 1800 Elo in Chess Without Engines
Google Astra's chess ability drew attention: on 09-07 thomasahle asked about its strength, and ivenger cited existing evaluations putting it around Elo 1500-1800. On 09-08 Mike Frank reported hands-on testing: to prevent "cheating" (downloading the Stockfish engine or chess databases), he verified the model could play without external tools, and its measured rating exceeded 1800.
Confirmed
- 09-07: thomasahle asked about Astra's chess level; ivenger replied existing evaluations put it at Elo 1500-1800.
- 09-08: Mike Frank measured Astra at over 1800 Elo with Stockfish and game databases disallowed.
- Mike Frank also noted that in the earlier tile-puzzle case, Astra could have written its own solver within minutes, suggesting its real capability may be underestimated.
Controversy
- iruletheworldmo questioned the value of chess benchmarks: a model could simply download Stockfish and beat Magnus Carlsen. Yet the benchmark remains watched precisely because it tests the model's own reasoning, which is why Frank emphasized the no-tools setup.
Why it matters
- Chess has become a representative test of frontier models' general reasoning, but methodology—whether external tools are disabled—determines credibility. This tool-free measurement gives a closer estimate of the model's true strength.
2026-09-07 ~ 2026-09-08 · 5 related posts
- Episode 1: Astra Loses to 1300-Elo Chess Bot, Then Wins with Ultra Mode and Visualization Tool(2026-09-06, 7 posts)
- Episode 2: GPT-6 Astra beats chess engine with raw model output, 41-move mate(2026-09-06, 3 posts)
- Episode 3: OpenAI's Astra Loses to 1800-Elo Chess Bot After Trapped Bishop Blunder(2026-09-06, 7 posts)
- Episode 4: Astra Scores Over 1800 Elo in Chess Without Engines(2026-09-07, 5 posts)
- Episode 5: AI Agent Astra Revamps Engine, Checkmates 1800-Elo Chess Bot in 31 Moves(2026-09-08, 6 posts)
Primary sources
- How Strong is Astra at Chess? User Asks — thomasahle · 2026-09-07
- Astra's Chess Elo Estimated Between 1500 and 1800 — ivenger · 2026-09-07
- Chess benchmarks questioned: a model could just download Stockfish and crush Magnus — iruletheworldmo · 2026-09-08
- [source] Testing Gemini's Astra at chess without engines or databases: rated above 1800 — MikePFrank · 2026-09-08
- [source] Testing Astra 'without cheating': chess rating above 1800, can code its own solver — MikePFrank · 2026-09-08