TogetherAI Open-Sources XoRL for Bitwise-Identical MoE RL Training and Inference

TogetherAI (@PandaAshwinee) released and open-sourced XoRL, a distributed RL training framework claiming zero (bitwise-identical) train-inference mismatch on large-scale MoE models, validated with ablation studies. It addresses a long-standing issue in asynchronous RL where computational inconsistencies between training and inference engines accumulate errors.

Confirmed

• XoRL is open-sourced; it uses a forked version of SGLang as the inference engine, strictly aligning training and inference computations including the kernel layer, achieving 0 train-inference error
• Validated on teaching Qwen3.6-35B-A3B to play Wordle: zero-mismatch training raised the solve rate from 63.9% to 77.4%
• The team published a detailed write-up covering the alignment method between the XoRL training engine and SGLang inference engine, with multiple ablations and a discussion of stable asynchronous RL training
• @marcbhargava relayed that the River API outperformed Tinker on RL tasks (same training code, better results), also achieving 0 train-inference mismatch on large MoE models

Why it matters

• Train-inference numerical mismatch has long plagued asynchronous RL systems, introducing hard-to-trace bias and hurting final performance; kernel-level bitwise alignment offers a reusable open-source solution
• Ablations quantify the payoff of zero mismatch (over 13 percentage points on Wordle solve rate), providing a reference baseline for future MoE RL engineering

2026-08-18 ~ 2026-08-18 · 5 related posts

• [source] TogetherAI open-sources XoRL: 0 train-infer mismatch for large MoE RL training — PandaAshwinee · 2026-08-18
• [source] Achieving 0 Train-Infer Mismatch for MoE RL, Boosting Wordle Performance — PandaAshwinee · 2026-08-18
• [source] Deep Dive: 0 Train-Infer Mismatch for Open-weight MoE RL — PandaAshwinee · 2026-08-18
• Open Source Method Eliminates Train-Inference Mismatch in MoE RL, Boosting Performance — PandaAshwinee · 2026-08-18
• River API outperforms Tinker in RL, enabling zero train-infer mismatch for MoEs — marcbhargava · 2026-08-18