Microsoft's ActiveSaddler adapts agent harness training scenarios, boosting Pass@1 by up to 7.5 points
dair_ai · x · 2026-10-04
A new paper from Microsoft and colleagues introduces ActiveSaddler, a method for optimizing agent harnesses.
- Problem: Current harness optimizers only change how the harness is updated while keeping training scenarios fixed, so feedback from tasks becomes uninformative as the harness improves.
- Method: ActiveSaddler groups recurring failures into failure patterns, treating each pattern as an arm in a non-stationary bandit. It tracks how much the harness still learns from each pattern and splits the budget between revisiting known weaknesses and discovering new ones, adapting scenarios alongside the harness.
- Results: With the same optimizer, test Pass@1 improves by 4.4 points on GAIA2 and 7.5 points on Terminal-Bench 2.0 versus a fixed scenario order.
More from coding & agent
- TanStack AI adds experimental text-to-GLB 3D model and prompt-to-motion animation generation — bennash · 2026-11-20
- Dev builds typing survival game with troops and skill trees in ~24 hours using agents and kaigen3d — hullabaloo22 · 2026-11-20
- Harmonic's Aristotle writes proven-correct fusion plasma controllers in C and Verilog in 4 hours — satnam6502 · 2026-11-20
- R3 Code: A Native Rust Mac Client That Runs All Your Coding Agents — jasonkneen · 2026-11-20
- zlib: 2.4K-star open-source CLI turns Z-Library into a skill for AI agents — udmrzn · 2026-11-20
- Andriy Burkov's True Positive Weekly #186: what to know when agents do the work — burkov · 2026-11-20