vLLM's bitwise-consistent RL: zero KL on Nemotron NVFP4, 91.3% throughput on DeepSeek-V4-Flash
DynamicWebPaige · x · 2026-10-15
Full results and tooling from vLLM's train-inference consistency work:
- DeepSeek-V4-Flash: diff = 0 at every step over 100 RL steps on 32 GB200 GPUs; rollout throughput 24.22 vs 26.53 output tokens/ms baseline (91.3%, 88.6% over first 89 steps).
- Nemotron-3.5-Lightning-30B-A3B (Mamba-2 hybrid + NVFP4): train–inference KL stays exactly 0 on a filtered SWE-bench Pro subset; training reward holds 0.56 → 0.59 while the baseline falls 0.56 → 0.22; rollout throughput 87.5% of baseline.
- The workflow is distilled into an open-source agent skill (vLLM PR #61551): find first diverging op → minimal repro → fix → re-verify → profile and optimize.
- DeepSeek-V4.1 has also reached 0 diff. Caveats: training metrics, not held-out benchmarks; end-to-end wall-clock speedup not yet measured.
More from coding & agent
- HYSET scores tool sets instead of single tools, lifting agent task success to 69.9% — DeepLearningAI · 2026-10-16
- Dan Wahlin shows how to verify agent-built features in GitHub Copilot's built-in browser — DanWahlin · 2026-10-16
- Amazon would drown in money with a headless shop for agents, argues dev — djcows · 2026-10-15
- poteto open-sources pstack, a skill set for rigorous autonomous coding agents — DanielLockyer · 2026-10-15
- Geoffrey Huntley proposes the Remix Model: closed to PRs, open to remixing — kieranklaassen · 2026-10-15
- E2B sandbox usage doubled to 2B in 5 months, 61% of traffic now from agents — FinanceYF5 · 2026-10-15