LittlePly training: BF16 autocast with Muon plus AdamW and gradient clipping
andreisavu · x · 2026-11-08
andreisavu's LittlePly chess agent thread reveals training details: BF16 autocast with FP32 parameter/gradient storage, Muon combined with AdamW, gradient clipping, and warmup/plateau/decay schedules tied to source progress. Initialization reuses a pretrained transformer body with fresh chess embeddings, adapted before full-network training; controlled playing tests must still establish the benefit of this initialization and the auxiliary objectives.
More from Research
- The meta-question as agent swarms scale: can models beat gradient descent at its own game? — JensHonack · 2026-11-08
- Why can LLMs solve PhD-level math but still lose at chess? Reddit asks — MesloubFC · 2026-11-08
- WebHarbor launches: 153 mirrored real websites and 3,518 tasks for GUI web agents — HuaxiuYaoML · 2026-11-08
- METR Progress per R&D Dollar Fell 63% Even as Yearly Gains Doubled — sebkrier · 2026-11-08
- GenrobotAI's DAS Dex 2.0 captures 21-DoF hand motion plus 980 tactile points for robot learning — CyberRobooo · 2026-11-08
- fordycelab proposes benchmarking binder-design algorithms with Kd measurements on 24,000-VHH libraries — anshulkundaje · 2026-11-08