vLLM achieves bitwise-consistent RL training: DeepSeek-V4-Flash hits 0 diff over 100 steps
vllm_project · x · 2026-10-15
vLLM published a new blog tackling a subtle RL training problem: same weights don't guarantee the same policy — inference optimizations alter the logprobs that training recomputes, breaking true on-policy RL.
The approach: the training forward pass calls vLLM's inference kernels directly, aligning FP8 GEMM, sparse-attention Top-K, FP8 KV-cache reads, MoE communication and CUDA Graph execution so the same token produces bitwise-identical results across validated batch sizes, phases and parallel layouts. Numerics are shared in the forward pass, with independent checks for the backward pass.
More from coding & agent
- HYSET scores tool sets instead of single tools, lifting agent task success to 69.9% — DeepLearningAI · 2026-10-16
- Dan Wahlin shows how to verify agent-built features in GitHub Copilot's built-in browser — DanWahlin · 2026-10-16
- Amazon would drown in money with a headless shop for agents, argues dev — djcows · 2026-10-15
- poteto open-sources pstack, a skill set for rigorous autonomous coding agents — DanielLockyer · 2026-10-15
- Geoffrey Huntley proposes the Remix Model: closed to PRs, open to remixing — kieranklaassen · 2026-10-15
- E2B sandbox usage doubled to 2B in 5 months, 61% of traffic now from agents — FinanceYF5 · 2026-10-15