SGLang's Delta Router Replay slashes sync stalls in Kimi K2 agentic RL training
hsu_byron · x · 2026-09-17
Engineers at Periodic Labs shared RL infrastructure details behind building Neon, focusing on minimizing training-inference mismatch.
Mechanism: To reduce train-inference mismatch, SGLang captures MoE routing decisions for each rollout during inference and 'replays' them during training.
The bottleneck: In agentic multi-turn tool-use settings, SGLang exports router decisions per decoding request. When any data-parallel rank finishes a conversation turn, all other ranks must wait for routing data export — exacerbated because the entire conversation's routing decisions are exported, not just the latest turn.
Delta Router Replay: Discovered during Kimi K2 RL setup by hsubyron and vwxyzjn, the fix caches previous turns' router decisions on the training client so only the delta (latest turn) gets exported, drastically cutting sync stalls.
More from Infra
- VC-Attention: training-free low-bit attention hits 1.9x on B200, beating FlashAttention-4 — xiuyu_l · 2026-09-17
- mlx.fast fixes speed-display bug: MLX kernels hit 80.6 tps, nearing 100% speedup milestone — HankYeomans · 2026-09-17
- Memory shortage hits checkout: Xiaomi raises phone prices 200-1,000 yuan as DRAM stock dips under 10 days — tengyanAI · 2026-09-17
- Burkov Slams OpenRouter Reliability: Fallback Models Fail Together — burkov · 2026-09-17
- Common Crawl puts crawl archives on Hugging Face Storage Bucket, with a getting-started guide — vanstriendaniel · 2026-09-17
- Anthropic Signs A$32B Queensland Data Center Deal, Claude's First Australian Footprint — ocean_protocol · 2026-09-17