A Qwen3-4B RL harness reproduction tests transfer from TREC 32K to 256K
ben_burtenshaw · x · 2026-07-23
A minimal reproduction of the “language model harnesses are compositional generalizer” idea using a small, reproducible setup.
- The author rebuilt the experiment with Qwen3-4B, TRL GRPO, and an OpenEnv REPL environment so others can tinker with it in one script on one node.
- The setup trains on TREC 32K and tests whether the learned harness transfers to TREC 256K and spam 128K.
- The goal is not a full faithful reproduction of the paper, but a simpler working version of the core mechanism: train on short contexts, keep long inputs outside the main context, and learn root decomposition with RL.
The follow-up note clarifies the assumptions: smaller model, frozen subcaller, shorter strategy training, lower subcall budgets, and no trajectory-similarity measurements yet.
Related event: Developer Creates Runnable RL Harness Prototype with Qwen3-4B(2 posts)→
More from coding & agent
- 'Let an agent run wild on vague goals and it makes your code worse 9 out of 10 times' — eptwts · 2026-10-27
- LangChain to demo the in-house design agent its team uses daily, this Wednesday — LangChain · 2026-10-27
- Biotech Agent Series Day 8: A Notebook Review Agent to Auto-Audit Lab Records — nlarusstone · 2026-10-27
- Photon Launches Agent-to-Agent Messaging in iMessage for Real AI Collaboration — alvelda · 2026-10-27
- Fake-paper meme shows an agent workflow for reverse engineering trading systems into a one-person quant fund — PtrPomorski · 2026-10-27
- Codex clones Linear's core from a one-page spec in 3 minutes 28 seconds — aakashgupta · 2026-10-27