A Qwen3-4B RL harness reproduction tests transfer from TREC 32K to 256K
ben_burtenshaw · x · 2026-07-23
A minimal reproduction of the “language model harnesses are compositional generalizer” idea using a small, reproducible setup.
- The author rebuilt the experiment with Qwen3-4B, TRL GRPO, and an OpenEnv REPL environment so others can tinker with it in one script on one node.
- The setup trains on TREC 32K and tests whether the learned harness transfers to TREC 256K and spam 128K.
- The goal is not a full faithful reproduction of the paper, but a simpler working version of the core mechanism: train on short contexts, keep long inputs outside the main context, and learn root decomposition with RL.
The follow-up note clarifies the assumptions: smaller model, frozen subcaller, shorter strategy training, lower subcall budgets, and no trajectory-similarity measurements yet.
Related event: Developer Creates Runnable RL Harness Prototype with Qwen3-4B(2 posts)→
More from coding & agent
- OneCLI lets AI agents work inside companies without storing passwords — ycombinator · 2026-07-23
- Mapping Open-Source AI with Codex Agents: Mask2Former Found Lagging Behind SOTA — NielsRogge · 2026-07-23
- Offloop says its multi-agent router delivers SOTA at 80% lower task cost — umesh_ai · 2026-07-23
- New tool manages fleets of agents across Claude Code, Codex and Cursor — dee_hw · 2026-07-23
- Dan Vega is prototyping Figma thumbnail concepts with Claude Code Skills — therealdanvega · 2026-07-23
- Databricks explains when to use Genie Agents, Knowledge Assistant, and a Supervisor — CautiousUse8597 · 2026-07-23