The Optimizer Is the Agent: Reasoning-Driven Search across Prompts, Programs, and ML Workflows
Junbo Li, Boyi Liu, Canwen Xu, Yite Wang, Yuxiong He, Zhangyang Wang, Qiang Liu, Zhewei Yao
COLM 2026
cs.AI
2026-08-07
One tool-using code agent optimizes prompts, programs, and ML configs, deciding what to evaluate, diagnose, edit, and restart; it beats specialized systems by 2-40% on 14 tasks.
Optimizing "text artifacts" (system prompts, programs, training configs) is a routine need. Existing approaches mostly rely on external search controllers such as evolutionary search, bandits, textual gradients, or Bayesian optimization. The LLM just proposes candidates; the real search strategy (which candidate to pick, when to switch direction, when to restart) is hand-designed and lives outside the model.
ReASearch is a minimal code agent with file I/O, Python/Bash execution, and a lightweight memory module. Each turn it receives the system prompt, the current optimization state, and its interaction history, then reasons about which tool to call next. The core loop is task-agnostic: all task-specific information enters through two interfaces, the tool set and the task-specific system prompt, so the same scaffold optimizes prompts, programs, and training scripts.
python exec is the most important tool, turning the agent from a pure text reasoner into a "computational reasoner" that writes analysis scripts, computes statistics, and locates failure modes. Long-horizon optimization relies on two mechanisms: context compression (a compaction tool summarizes history past a threshold) and a persistent lessons.md file recording what worked, what failed, and what to try next.
| Task | Baseline | Best specialized | ReASearch |
| Terminal-Bench 2.0 (prompt) | 35.6 | GEPA 42.2 | 53.3 |
| ARC-AGI-2 test (program) | -- | AdaEvolve 12.5% | 50.0% |
| IMG-100 (ML workflow) | 63.5 | Claude Code 78.6 | 84.0 |
On prompt optimization it beats GEPA across all four tasks; on program evolution it matches or beats AdaEvolve and in some cases improves on human best-known results (Circle Packing); on ML workflows it beats Claude Code's AutoResearch on four of five tasks (statistical tie on NanoGPT) and lifts a Kaggle crypto-market prediction submission from rank 36 to 6. A full run costs under $20 in API usage.
The authors' main claim is that search behaviors previously hand-coded (candidate verification, hypothesis-driven diagnosis, when to revert, overfitting detection, accumulating lessons across rounds) emerge from the agent's reasoning rather than from an external controller. The practical value is one scaffold replacing a pile of specialized optimizers, at lower token cost.
The authors concede in Appendix F that the edge over Claude Code partly comes from the harness re-emitting the current search state (best result, recent experiments, lessons, stagnation warnings) into the prompt every turn, not only from the instructions themselves. So the credit split between "the agent reasons out search" and "the scaffold orchestrates state" is not clean. Baselines are limited to GEPA, AdaEvolve, and Claude Code. The "emergent behaviors" are largely qualitative trajectory retellings, not quantitative metrics. At $20 per run across 14 tasks and cross-validation folds, total cost is not trivial.