Why do commercial LLMs still fail at chess despite all that pretraining?
alex_peys · x · 2026-07-29
The author argues that chess is still a useful benchmark for LLMs because commercial models continue to perform poorly at it despite having likely seen huge amounts of chess material in pretraining.
Their key questions are:
- if the raw knowledge is already in the model, why is chess still hard?
- fine-tuning with supervised learning or RL on chess would obviously make models much stronger than humans
- but is there a way to extract chess skill without brute-force task-specific rewards or behavior cloning?
The post is less a claim than a research-style question about what pretraining actually contains and how capabilities can be unlocked.
Related event: LLMs Still Struggle with Chess Despite Pre-training(3 posts)→
More from Research
- Tsinghua professor's ApexIntelligence raises $50M to build self-evolving foundation models — DeryaTR_ · 2026-10-16
- Fluids Don't Compute—They Contribute the Nonlinear Step, Sandberg Argues — anderssandberg · 2026-10-16
- Anders Sandberg on fluid computation: error budgets tighten quadratically over time — anderssandberg · 2026-10-16
- How the fluid computation works: force field advects particle to 0/1 boxes — anderssandberg · 2026-10-16
- FlashAttention creator Tri Dao joins Radical Numerics as scientific advisor — exnx · 2026-10-16
- SAIR Launches Open Math OS, Offering Free Compute and 100+ Tools for Community-Owned Math AI — Thom_Wolf · 2026-10-16