OPAL: a coding agent rewrites your optimizer mid-training, live
vincesitzmann · x · 2026-10-22
Vincent Sitzmann introduces OPAL (OPtimizer with an Agent in the Loop), an open-source PyTorch wrapper (MIT, pip-installable) that lets a coding agent like Claude Code keep rewriting the optimizer and LR schedule while training runs:
- Detects loss spikes, zeros them out, and watches for them afterward; notices overshooting the minimum, resets momentum, and rolls back a few steps.
- Designs LR schedules as sophisticated as those only super-late runs like DeepSeek could afford, set per-module (block matrices, adaLN modulation, embeddings).
- Author's bet: the era of fixing optimizer/LR schedule before a run and never touching it again is over — reactive optimizers are like hiring a DL expert to watch training full-time.
- Usage: wrap your PyTorch optimizer with one line and plug in your agent.
Related event: EPFL Releases OPAL, an Optimizer Rewritten by Agents in the Loop(2 posts)→
More from coding & agent
- One researcher pretrains 100B-500B coding model in 3.5 months on under $10M, agents write 99% of code — RobertTLange · 2026-10-22
- 'Boss of superintelligence': agent autonomously debugs CI, asks permission for computer use — McDonaghMatthew · 2026-10-22
- Agent rejects an approved design review, reruns it with a skill-capable agent type — umeshonai · 2026-10-22
- How to stop Claude Code from torching your API budget overnight: gateway routing explained — Lord_Alpha01 · 2026-10-22
- Four Gemini agents beat Minecraft's Ender Dragon in 27 hours — dying 605 times — krishnan · 2026-10-22
- Dev take: AI coding frees you to think deeply — and exposes that most can't — KevinNaughtonJr · 2026-10-22