Microsoft's ActiveSaddler: automated curriculum learning for agent harness optimization, +4-7pt Pass@1
rohanpaul_ai · x · 2026-10-05
Problem
Existing automated harness optimization methods tune how to update prompts, tool interfaces, and control logic, but keep the training scenarios that generate feedback fixed. As the harness evolves, the most useful scenarios change too.
Approach: ActiveSaddler
- Models the evolving curriculum as a non-stationary bandit with dynamically instantiated optimization targets
- Abstracts recurring failures into reusable failure-pattern arms
- Estimates potential learning progress per pattern, adaptively balancing revisiting known weaknesses vs. exploring unseen scenarios
- Continually updates the discovered failure patterns and priorities so curriculum co-evolves with the harness
Results
On GAIA2 and Terminal-Bench 2.0, consistently finds stronger harnesses, improving test Pass@1 by 4.4 and 7.5 percentage points.
More from coding & agent
- Hanging Context: keyless MCP server for citable news lands in Claude's directory — bshivarthy · 2026-10-24
- Devs debate codex: remote control mode called 'miles better' in workflow comparison — _sholtodouglas · 2026-10-24
- Dev Vibe-Codes MOSS Image Studio, a Friendlier Browser Frontend for ComfyUI — Roorem10 · 2026-10-24
- He Built Custom TV Launchers for Guests and His Backyard With Claude Code — rephlektiv · 2026-10-24
- Claude skill claims to replace Higgsfield Genjutsu with Seedance 2.5 video editing — aziz4ai · 2026-10-24
- Daniel Miessler: code is the lossy output — the primary artifact is the fully articulated ideal in language — JensHonack · 2026-10-24