OPAL: an LLM agent rewrites the optimizer's source code during training
RobertTLange · x · 2026-10-23
OPAL (OPtimizer with an Agent in the Loop) puts an LLM agent inside the training loop — not an outer-loop hyperparameter tuner, it watches training dynamics and rewrites the optimizer's source code mid-run.
The interface is two agent-written files:
- analysis: probes the agent writes itself to pull whatever stats it wants (per-group grad norms, update/weight ratios, cosine similarity between consecutive gradients, spike detectors) — raw weights would flood its context, so it chooses what to inspect;
- updater: the update rule run every step. Since the interface is code, anything can change: per-group LR schedules (including raising LR mid-run), AdamW→Muon on specific tensors, bf16 vs fp32 per tensor, momentum resets, clipping, gating bad batches, even A/B tests on top.
More from coding & agent
- Garry Tan endorses Aside Browser as the fix for agents getting blocked on the web — garrytan · 2026-10-23
- MCP server turned rogue 30 days after approval, prompting open-source mcp-guard lockfile tool — ErgoniaWorks · 2026-10-23
- Test: Coding Agents Won't Use an MCP Tool Unless AGENTS.md Mentions It — colinmcnamara · 2026-10-23
- Open-Source Tennis Models Plus Claude Turn Video Into a World-Class Coach — tweetsatpreet · 2026-10-23
- awesome-ai-apps: 132 Free Open-Source AI Agent and RAG Projects on GitHub — Arindam_1729 · 2026-10-23
- Google's Agents CLI Adds Extension System, Ships First-Party LangChain Support — rseroter · 2026-10-23