New paper: LM harnesses are compositional generalizers, +26% on long tasks
a1zhang · x · 2026-11-20
Researcher a1zhang published "LM Harnesses are Compositional Generalizers": with stronger models in 2026, harnesses now serve to compose existing LLM capabilities and generalize beyond training environments rather than crutches for weak models.
- Harness abstractions (RLMs, CodeAct, MemGPT) enable generalization over task length, strategy, and recursion depth by keeping every LLM call locally in-distribution (LID), enabling data-efficient training for long tasks.
- For multi-day rollouts, surrogate harness training trains individual LLM calls instead of full harness rollouts.
- Training Qwen3.8-27B on fully synthetic data (tasks generated by Claude Opus 5.5) improves relative performance 26% where naive RL fails to generalize.
- Follow-ups: "trajectory-as-a-prompt" causes context rot; recursive harness invocation (origin of the RLM name) trains via tail recursion, lifting RLM-27B long-task average from 30.5 to 58.5.
Related event: MIT Paper: LM Harnesses Are Compositional Generalizers, Not Crutches(10 posts)→
More from coding & agent
- Build a 'company brain': one shared knowledge base for your whole team's AI — VibeMarketer_ · 2026-11-21
- LangChain ships four updates: LangSmith Agent Platform, trace data controls, Tool Gateway — LangChain · 2026-11-21
- Stanford CS329Z: Full AI Agents Course Free With Slides, Videos, Homework — stanfordnlp · 2026-11-21
- Grok Bot Gets Its Own Email Addresses to Sign Up and Handle Promos for You — elonmusk · 2026-11-21
- CapCut × ChatGPT plugin turns chat into a video generator and editor — SimplyAnnisa · 2026-11-21
- Princeton & Meta's Strategist evolves inference strategies, beating baselines on 24/30 benchmarks at 39% lower cost — 141_1337 · 2026-11-21