Qwen Learns to Train Qwen
DanAiTuning · reddit · 2026-07-14
The author states they trained Qwen3.6-35B-A3B into a model that "uses RL to train other models": upon receiving a task, it writes the entire training job—including environment, rewards, datasets, and hyperparameters—and submits it to real GPUs for execution.
Training Method
- Inner loop: Trains small Qwen models (0.6B / 1.7B) on real GPUs using prime-rl and verifiers
- Outer loop: Uses Tinker to perform RL on the trainer itself, with "the trained small model becoming stronger" as the reward
- A total of 6 task categories were designed, with 1 category entirely reserved for generalization testing
Results
- Outer episode reward increased from 0.0 to a peak of 0.63, backed by roughly 1750 real training jobs
- Transfer occurred on unseen task families, indicating it isn't just rote memorization
- The trainer learned to use the 1.7B base more often and began truly utilizing the hyperparameter space
- Total cost was approximately $1.3k, with a single inner-loop training costing about $0.13–0.30
The author open-sourced the repository, task families, reward code, GPU scheduling, and a retrospective.
Related event: Developer Uses RL to Train Qwen to Train Other AI Models(2 posts)→
More from Research
- Nat Lambert shares a reading list on synthetic data and agentic SFT data — natolambert · 2026-07-22
- Lightwheel AI Launches SimReadyGen: Text-to-Physics-Accurate Robot Sim Assets — ZeYanjie · 2026-07-22
- PNAS special issue examines copyright, governance, and AI in the legal system — chrmanning · 2026-07-22
- WeirdChat catalogs strange model behaviors from more than 100 million sampled responses — JacobSteinhardt · 2026-07-22
- New agentic benchmark shows AI managers escalate to coercion and fake success — Jasmine Brazilek · 2026-07-22
- Ai2’s Asta adds one-click handoff and self-checking deep paper search — allen_ai · 2026-07-22