SkillOpt: Executive Strategy for Self-Evolving Agent Skills
Yifan Yang, Ziyang Gong, Weiquan Huang, Qihao Yang, Ziwei Zhou, Zisu Huang, Yan Li, Xuemei Gao, Qi Dai, Bei Liu, Kai Qiu, Yuqing Yang, Dongdong Chen, Xue Yang, Chong Luo
cs.AI, cs.CL
2026-05-23
SkillOpt trains a frozen agent's skill with bounded gated edits. GPT-5.5 rises +23.5 in direct chat, +24.8 in Codex, +19.1 in Claude Code; best or tied on all 52 cells.
Most agent skills are still written by hand, generated in one shot, or revised from failure traces without a step size, a validation split, or a way to reject a bad edit. Closed models cannot take weight updates; open ones make fine-tuning expensive. The missing piece is an optimizer for the skill document itself.
SkillOpt treats that document as the external state of a frozen agent and trains it with the same controls used in weight space: batch evidence, a learning-rate budget, a held-out gate, and a memory of rejected steps. At deployment the extra artifact is a 300 to 2,000 token bestskill.md. The target model and harness stay fixed.
The frozen target runs a rollout batch with the current skill and returns scored trajectories. A separate optimizer model splits successes from failures, reflects in minibatches, and proposes structured add, delete, and replace edits. After merge and rank, only the top Lt edits are applied (default 4, cosine decay to 2). A candidate is accepted only if the held-out selection score strictly rises; ties are rejected. Rejected edits and the score drop they caused sit in an epoch-local buffer so later reflections do not repeat them.
At epoch end a slow update reruns the same training items under the previous and current skills, groups them into improvements, regressions, persistent failures, and stable successes, and writes a protected slow-update field that step-level patches cannot overwrite. An optimizer-side meta skill records which edit patterns helped; it never ships with the student. The same bestskill.md is injected through adapters into direct chat, Codex CLI, and Claude Code. Default run: four epochs, rollout batch 40, reflection minibatch 8.
Six benchmarks (SearchQA, SpreadsheetBench, OfficeQA, DocVQA, LiveMathematicianBench, ALFWorld) times seven target models times three harnesses give 52 cells. SkillOpt is best or tied on all 52. Baselines cover no skill, human skill, one-shot LLM skill, Trace2Skill, TextGrad, GEPA, and the harness-side EvoSkill.
On GPT-5.5 direct chat the six-benchmark mean moves from 58.8 to 82.3, +23.5 points. An oracle that picks the strongest competing method per cell still only reaches 76.9, 5.4 points behind.
| Benchmark | No skill | SkillOpt | Δ |
| SearchQA | 77.7 | 87.3 | +9.6 |
| SpreadsheetBench | 41.8 | 80.7 | +38.9 |
| OfficeQA | 33.1 | 72.1 | +39.0 |
| DocVQA | 78.8 | 91.2 | +12.4 |
| LiveMath | 37.6 | 66.9 | +29.3 |
| ALFWorld | 83.6 | 95.5 | +11.9 |
GPT-5.5 gains +24.8 under Codex and +19.1 under Claude Code. Across seven direct-chat targets the mean lift is about +17.6; weaker students gain more in relative terms. GPT-5.4-nano goes from 34.3 to 69.4 on ALFWorld and from 30.8 to 80.2 on DocVQA. Qwen3.5-4B spreadsheet accuracy goes from 9.3 to 23.9.
Transfer holds. A spreadsheet skill trained on GPT-5.4 stays positive on mini and nano (+9.4 / +3.0). A Codex-trained spreadsheet skill dropped into Claude Code moves the no-skill 22.1 to 81.8, +59.7, slightly above the in-domain Claude Code SkillOpt score of 80.4. An OlympiadBench skill, untuned, adds 3.7 points on Omni-MATH for GPT-5.4.
The sharpest ablation: drop both slow update and meta skill, and SpreadsheetBench falls from 77.5 to 55.0. Drop the rejected-edit buffer and the three-benchmark scores fall 1.6, 4.6, and 2.4. Unbounded rewrite without a learning-rate cap scores 84.6 / 75.7 / 57.3, below bounded edits. Only 1 to 4 edits are actually committed to bestskill.md (median 2.5). OfficeQA's +39.0 and LiveMath's +29.3 each come from a single accepted edit. Final documents run 379 to 1,995 tokens. The surviving rules are procedural, not instance-specific: on spreadsheets the skill tells the agent to inspect workbook structure and formulas, then write evaluated static values across the full target range instead of waiting for Excel to recalculate.
For closed models and teams that will not fine-tune per domain, this is an auditable adaptation path: burn rollouts offline with a stronger optimizer, then ship a readable markdown file. Gains concentrate on brittle procedure (spreadsheets, office docs, answer format). Factual QA is already near ceiling, so the lift is smaller. EvoSkill already takes Codex spreadsheets from 27.5 to 67.5; SkillOpt continues to 85.0. The gap is bounded step size, a validation gate, and rejected-edit memory, not a longer prompt. This is prompt optimization run as controlled training, not a new model.
The loop needs automatic scores. Open-ended or subjective tasks, or expensive human judges, leave the gate without a reliable signal. Training burns rollouts and optimizer calls, which may not pay off for one-off jobs. One compact domain skill may be too thin for heterogeneous procedure sets. Learned rules still smell of the training distribution, so far-away models, harnesses, and task families still need held-out checks. Dollar API cost is not reported, only training tokens: SearchQA and DocVQA cost 37.9M and 46.4M tokens per test point, while spreadsheets cost 0.6M. ALFWorld is blank under Codex and Claude Code because the standard adapters cannot hold persistent embodied interaction.