Qwen Learns to Train Qwen
DanAiTuning · reddit · 2026-07-14
The author states they trained Qwen3.6-35B-A3B into a model that "uses RL to train other models": upon receiving a task, it writes the entire training job—including environment, rewards, datasets, and hyperparameters—and submits it to real GPUs for execution.
Training Method
- Inner loop: Trains small Qwen models (0.6B / 1.7B) on real GPUs using prime-rl and verifiers
- Outer loop: Uses Tinker to perform RL on the trainer itself, with "the trained small model becoming stronger" as the reward
- A total of 6 task categories were designed, with 1 category entirely reserved for generalization testing
Results
- Outer episode reward increased from 0.0 to a peak of 0.63, backed by roughly 1750 real training jobs
- Transfer occurred on unseen task families, indicating it isn't just rote memorization
- The trainer learned to use the 1.7B base more often and began truly utilizing the hyperparameter space
- Total cost was approximately $1.3k, with a single inner-loop training costing about $0.13–0.30
The author open-sourced the repository, task families, reward code, GPU scheduling, and a retrospective.
Related event: Developer Uses RL to Train Qwen to Train Other AI Models(2 posts)→
More from Research
- HarnessMM cuts agent tokens per task by up to 88% while raising success rates — omarsar0 · 2026-10-26
- Ben Recht: The world needs a truly open language model, and it's easier to build than you think — beenwrekt · 2026-10-26
- New Blog Breaks Black-Box Distillation Into Five Practical Steps — chrisrohlf · 2026-10-26
- DetectorScope turns black-box AI text detectors into auditable ones with sparse autoencoders — _akhaliq · 2026-10-26
- OPST paper shows LLMs self-improve with zero human data, guided only by topic descriptions — _akhaliq · 2026-10-26
- Tencent's HiSTEVE 1.0: hierarchical VLA agent plays Minecraft with native keyboard and mouse, hits 85% SOTA — _akhaliq · 2026-10-26