EPFL's OPAL Puts a Coding Agent Inside the Training Loop

A team at EPFL led by Vincent Sitzmann (members include @SimulatedAnneal, @jaydenteoh, and Jia-Hao Liang) released OPAL (OPtimizer with an Agent in the Loop): a PyTorch optimizer wrapper that lets coding agents like Claude Code continuously monitor training and intervene on the fly. The authors' core judgment: the era of pre-selecting an optimizer and learning-rate schedule and running to completion untouched may be over.

Confirmed

OPAL can automatically detect loss spikes during training and suppress them back to zero, then keep guarding against similar spikes (m1, m4)
Demos show the agent noticing optimization steps overshooting a minimum, rolling back several steps, resetting momentum, then resuming monitoring; experiments include corruption-resilient ResNet training on CIFAR-10, with leading performance (m3, m4)
OPAL can automatically design highly intricate learning-rate schedules; the authors say such schedules were previously affordable only for the highest-priority, very-late-stage training runs at places like DeepSeek, and ordinary users can now get equal or greater sophistication (m2, m6)
The team has packaged the concept and published the accompanying writing (m5)

Why it matters

Optimizers and learning-rate schedules have long been the parts of deep learning training most dependent on human expertise and compute budgets. By putting an agent in the training loop, OPAL makes "reactive" optimization—constant vigilance with immediate intervention—a general capability. If it works as described, it will sharply lower the barrier to high-quality schedules and change how automated training pipelines can be. Current information comes mainly from the authors' own accounts and demos; independent replication and validation on larger-scale training remain to be seen.

2026-10-22 ~ 2026-10-22 · 8 related posts

Primary sources

OPAL: a coding agent rewrites your optimizer mid-training, live — vincesitzmann · 2026-10-22 00:14 (UTC+08:00)
OPAL optimizer auto-detects and suppresses loss spikes, beating Muon and AdamW under data corruption — vincesitzmann · 2026-10-22 00:14 (UTC+08:00)
OPAL: researchers package 'optimizer with an agent in the loop' concept — kingslyj · 2026-10-22 00:35 (UTC+08:00)

OPAL in action: agent resets momentum and rolls back after overshooting — vincesitzmann · 2026-10-22
[source] OPAL optimizer auto-detects and suppresses loss spikes, beating Muon and AdamW under data corruption — vincesitzmann · 2026-10-22
OPAL auto-designs DeepSeek-grade learning rate schedules for anyone — vincesitzmann · 2026-10-22
[source] OPAL: a coding agent rewrites your optimizer mid-training, live — vincesitzmann · 2026-10-22
OPAL: agent-in-the-loop reactive optimizers may end fixed learning-rate schedules, says Sitzmann — vincesitzmann · 2026-10-22
[source] OPAL: researchers package 'optimizer with an agent in the loop' concept — kingslyj · 2026-10-22
OPAL: an agent rewrites optimizer source code during training, beating static recipes — vincesitzmann · 2026-10-22

1 near-duplicate retellings: vincesitzmann