A Qwen3-4B RL harness reproduction tests transfer from TREC 32K to 256K
ben_burtenshaw · x · 2026-07-23
A minimal reproduction of the “language model harnesses are compositional generalizer” idea using a small, reproducible setup.
- The author rebuilt the experiment with Qwen3-4B, TRL GRPO, and an OpenEnv REPL environment so others can tinker with it in one script on one node.
- The setup trains on TREC 32K and tests whether the learned harness transfers to TREC 256K and spam 128K.
- The goal is not a full faithful reproduction of the paper, but a simpler working version of the core mechanism: train on short contexts, keep long inputs outside the main context, and learn root decomposition with RL.
The follow-up note clarifies the assumptions: smaller model, frozen subcaller, shorter strategy training, lower subcall budgets, and no trajectory-similarity measurements yet.
Related event: Developer Creates Runnable RL Harness Prototype with Qwen3-4B(2 posts)→
More from coding & agent
- Gergely Orosz: Shipping 10x PRs With AI Agents, Sites Fill With Small Regressions — ducha_aiki · 2026-09-11
- Same Echo Maze prompt, three frontier models: all passed visually but shipped the same hidden bug — eyishazyer · 2026-09-11
- Astra storyboards plus Minimax H3 per-shot generation boost video success rates — Hailuo_AI · 2026-09-11
- Codex tip: use Sol with Astra and Luna sub-agents to save usage — pvncher · 2026-09-11
- agents-best-practices: a provider-neutral Agent Skill for designing and auditing agentic harnesses — tom_doerr · 2026-09-11
- Cognition's SWE-2 uses a KKT duality argument in RL to shift the effort Pareto curve — YouJiacheng · 2026-09-11