LittlePly author: playing tests must fix checkpoints and assistance settings
andreisavu · x · 2026-11-08
In andreisavu's LittlePly thread, the author details evaluation methodology: playing tests must fix the checkpoint, decoder, assistance and clock settings, using Sunfish and a fixed earlier model as references. He stresses prediction loss doesn't measure playing strength, small match scores are noisy, draws must separate locally-verified automatic draws from claimable ones, and bounded engine assistance provides no formal guarantee.
More from Research
- Scaling May Be Superlinear: LessWrong Debate on Steepening AI Progress Curves — teortaxesTex · 2026-11-08
- Reviewers reward flaw-free work, users reward useful work — why peer review punishes innovation — wzenus · 2026-11-08
- Math community debates Glazer's proof and OpenAI's H10 paper: slop need not compound — littmath · 2026-11-08
- Stanford paper: more info helps some LLM trading agents, wrecks others — rohanpaul_ai · 2026-11-08
- Neel Nanda pushes back on LRH critique: one steering vector, four norms, four behaviors — NeelNanda5 · 2026-11-08
- AI-generated Community Note fact-checks weight-space programming paper, author clicks No — DimitrisPapail · 2026-11-08