Async GRPO Setup Left Some Training Runs Without the Full Data Mixture
yb2698 · x · 2026-11-09
The author details how their training setup (fixed number of steps plus async GRPO) meant not every run saw the entire training mixture, and lists the task ranges each model actually encountered:
- LFM2.5-24B-A2B: 435–1,212 tasks, covering both SmolDataEnvs and traj2envs (multiple passes for pi and opencode)
- LFM3-1.7B: 474–1,288, partly reaching traj2envs
- TwIL-8B: 116–159, SmolDataEnvs only
Each step trains on as many finished rollouts as fit its token budget. Longer rollouts (TwIL's long reasoning, the hermes harness) mean fewer tasks per step, and tasks ran in a fixed order, so some runs never reached traj2envs.
More from Research
- Tencent releases TailorRig: 39,073 rigged humanoid meshes for auto-rigging research — _akhaliq · 2026-11-10
- P²-MOPD uses expert weights as parameter priors to halve multi-teacher distillation steps — _akhaliq · 2026-11-10
- KlingAI releases LongBanana: 90-task benchmark for multi-reference image synthesis — _akhaliq · 2026-11-10
- Satellite Image Registration Still Uses Classical Cross-Correlation; Modern Matchers Lack a Benchmark — gabriberton · 2026-11-10
- Epoch AI researcher: most 'algorithmic progress' in frontier AI may come from better data — S_OhEigeartaigh · 2026-11-10
- Qdrant presents vector index-native relevance feedback in production at CIKM Rome — qdrant_engine · 2026-11-10