Nebius auto-tuner speeds RL training up to 5x on fixed GPU budgets

demian_ai · x · 2026-11-05

Nebius engineers (Di Jin, demianai, Mhr1036) detail how their internal RL auto-tuner delivered 1–5x faster RL training steps on fixed GPU budgets, with a median speedup of 3.5x across four models and two workloads.

Key insight: doubling inference-engine throughput in a benchmark didn't speed up RL training at all — the time saved on model calls was eaten by sandboxes. In another job, fixing their own CPU serving code saved 444 seconds per training step.
The next big gain can hide in request handling, sandbox setup, or stage overlap, and every successful change moves the bottleneck elsewhere.
The auto-tuner makes optimization repeatable: profile, forecast, test, measure again.
The write-up includes five surprising lessons from 210 runs, including why faster inference can leave RL just as slow.

Original post →

Nebius auto-tuner speeds RL training up to 5x on fixed GPU budgets

More from Infra