Surrogate harness training: RL-training long-horizon agents entirely on synthetic environments
a1zhang · x · 2026-11-20
Alex Zhang presents surrogate harness training from the RLM paper, tackling the problem that very long tasks (multi-day rollouts, sparse rewards) are nearly impossible to RL-train directly.
- The trick exploits composition: isolate the orchestrator LLM's trajectory inside the harness and train it in a fully synthetic environment — with sufficient context offloading and programmatic tool calling, tools and sub-calls can be fake, making rollouts cheap — then transfer to the real environment.
- The method composes recursively, chaining learned capabilities as tail-calls (Solve A→B→C→D), each stage a full harness trajectory.
This offers a viable path to RL-train expensive long-horizon agent tasks with synthetic data.
Related event: MIT Paper: LM Harnesses Are Compositional Generalizers, Not Crutches(10 posts)→
More from coding & agent
- Build a 'company brain': one shared knowledge base for your whole team's AI — VibeMarketer_ · 2026-11-21
- LangChain ships four updates: LangSmith Agent Platform, trace data controls, Tool Gateway — LangChain · 2026-11-21
- Stanford CS329Z: Full AI Agents Course Free With Slides, Videos, Homework — stanfordnlp · 2026-11-21
- Grok Bot Gets Its Own Email Addresses to Sign Up and Handle Promos for You — elonmusk · 2026-11-21
- CapCut × ChatGPT plugin turns chat into a video generator and editor — SimplyAnnisa · 2026-11-21
- Princeton & Meta's Strategist evolves inference strategies, beating baselines on 24/30 benchmarks at 39% lower cost — 141_1337 · 2026-11-21