Astra plays chess above 1800 Elo without engines or game databases

Google's Astra model is drawing attention for its chess strength: in a hands-on test with external tools disabled, Mike Frank measured a rating above 1800 — a solid amateur level — suggesting today's frontier models already have considerable cross-domain reasoning ability.

Confirmed

On 09-07, user thomasahle posted asking about Astra's chess strength; the same day, ivenger replied that existing evaluations put Astra at roughly Elo 1500-1800.
On 09-08, Mike Frank shared his test results: to prevent the model from "cheating" (downloading the Stockfish engine or a chess database), he specifically verified the model's ability without external tools, and measured a rating above 1800.
Mike Frank also noted that in the earlier tile-puzzle case, Astra could have written its own solver in minutes, implying the model's real capability may be underestimated.

Controversy

The same day, user iruletheworldmo questioned the meaning of chess benchmarks: a model could simply download a top engine like Stockfish and beat world champion Magnus Carlsen; yet the benchmark still draws attention precisely because it tests the model's own reasoning ability — which is why Frank's test emphasized "no cheating."

Why it matters

Chess has become a representative testbed for frontier models' general reasoning, but the methodology (whether external tools are allowed) directly determines how trustworthy the results are. By ruling out engines and opening-book databases, this test offers data closer to the model's true chess strength.

2026-09-07 ~ 2026-09-08 · 5 related posts

Primary sources

Testing Gemini's Astra at chess without engines or databases: rated above 1800 — MikePFrank · 2026-09-08 20:37 (UTC+08:00)
Testing Astra 'without cheating': chess rating above 1800, can code its own solver — MikePFrank · 2026-09-08 20:39 (UTC+08:00)
Astra's Chess Elo Estimated Between 1500 and 1800 — ivenger · 2026-09-07 18:42 (UTC+08:00)

How Strong is Astra at Chess? User Asks — thomasahle · 2026-09-07
[source] Astra's Chess Elo Estimated Between 1500 and 1800 — ivenger · 2026-09-07
Chess benchmarks questioned: a model could just download Stockfish and crush Magnus — iruletheworldmo · 2026-09-08
[source] Testing Gemini's Astra at chess without engines or databases: rated above 1800 — MikePFrank · 2026-09-08
[source] Testing Astra 'without cheating': chess rating above 1800, can code its own solver — MikePFrank · 2026-09-08