87M-parameter chess model: 12 decoder layers, chess tokens, tied embeddings
andreisavu · x · 2026-11-08
Parts 10-11 of the chess transformer thread detail the model architecture and supervision:
- Architecture: 87M parameters, 12 decoder layers, width 768, 12 attention heads, 512-token context, with a chess-specific token vocabulary and tied input/output embeddings.
- Supervision: ordinary next-move targets are weighted equally regardless of game outcome, while value is supervised by game results — policy and value signals serve separate roles.
More from Research
- The meta-question as agent swarms scale: can models beat gradient descent at its own game? — JensHonack · 2026-11-08
- Why can LLMs solve PhD-level math but still lose at chess? Reddit asks — MesloubFC · 2026-11-08
- WebHarbor launches: 153 mirrored real websites and 3,518 tasks for GUI web agents — HuaxiuYaoML · 2026-11-08
- METR Progress per R&D Dollar Fell 63% Even as Yearly Gains Doubled — sebkrier · 2026-11-08
- GenrobotAI's DAS Dex 2.0 captures 21-DoF hand motion plus 980 tactile points for robot learning — CyberRobooo · 2026-11-08
- fordycelab proposes benchmarking binder-design algorithms with Kd measurements on 24,000-VHH libraries — anshulkundaje · 2026-11-08