A Qwen3-4B RL harness reproduction tests transfer from TREC 32K to 256K
ben_burtenshaw · x · 2026-07-23
A minimal reproduction of the “language model harnesses are compositional generalizer” idea using a small, reproducible setup.
- The author rebuilt the experiment with Qwen3-4B, TRL GRPO, and an OpenEnv REPL environment so others can tinker with it in one script on one node.
- The setup trains on TREC 32K and tests whether the learned harness transfers to TREC 256K and spam 128K.
- The goal is not a full faithful reproduction of the paper, but a simpler working version of the core mechanism: train on short contexts, keep long inputs outside the main context, and learn root decomposition with RL.
The follow-up note clarifies the assumptions: smaller model, frozen subcaller, shorter strategy training, lower subcall budgets, and no trajectory-similarity measurements yet.
Related event: Developer Creates Runnable RL Harness Prototype with Qwen3-4B(2 posts)→
More from coding & agent
- Cognition's SWE-2 uses a KKT duality argument in RL to shift the effort Pareto curve — YouJiacheng · 2026-09-11
- First-ever Three.js Conference lands in Paris, with a panel on AI-shortened design workflows — OdinLovis · 2026-09-11
- Data engineering, not agent frameworks, is the real bottleneck for enterprise AI agents — dhruv2038 · 2026-09-11
- RTK Terminal Compression Cuts Tokens but Leaves Your AI Coding Bill Unchanged — Bartaseth · 2026-09-11
- GPT-6 Astra beats Factorio with enemies in 44 in-game hours at ~$4,500 API cost — liminal_bardo · 2026-09-11
- Investment Analyst Asks How to Build a Claude-Based Diligence Agent Stack — Careless_Tie2286 · 2026-09-11