Unified FP8 in Training and Rollout Speeds Up RL by 16%
joecole · x · 2026-07-30
A new paper Jet-RL from NVIDIA, MIT, and other institutions investigates the stability of using FP8 quantization during Reinforcement Learning (RL) training for Large Language Models (LLMs).
- The Bottleneck: The rollout phase typically consumes over 70% of total RL training time. The common practice of using BF16 for training and FP8 for rollout introduces significant numerical mismatch, leading to severe instability and accuracy collapse.
- The Jet-RL Solution: The framework proposes a unified FP8 precision flow for both training and rollout, minimizing numerical discrepancies and avoiding inefficient inter-step calibration.
- Performance: Experiments show that this approach achieves up to a 33% speedup in the rollout phase, a 41% speedup in the training phase, and a 16% end-to-end speedup over BF16 training while maintaining robust accuracy.
More from Infra
- SkyRL ships Day-0 RL support for Beam with +62% LoRA decode throughput in vLLM — vllm_project · 2026-10-16
- DFlash2 delivers up to 3.03x decode-only speedup for Beam across five benchmarks — vllm_project · 2026-10-16
- vLLM's Beam serving kernels cut GB200 forward time from 8.15 to 6.47 ms at TP8 batch 128 — vllm_project · 2026-10-16
- Reflection's 501B MoE Beam hits 463 tok/s on 8x GB200 with NVFP4 at day 0 — zhyncs42 · 2026-10-16
- Nemotron 3.5 Bolt runs 16 concurrent streams on a single DGX Spark — NVIDIAAI · 2026-10-16
- Why your computer can't store 0.2: an IEEE 754 primer — Franc0Fernand0 · 2026-10-16