Modal Trains RL on Mixed Accelerators, Cutting Cost While Matching Homogeneous Stability
charles_irl · x · 2026-10-14
Key point: RL is far more elastic if rollout FLOPs can come from any hardware.
- Modal notes that RL training of large models typically runs on a fixed homogeneous cluster of the latest accelerators for numerical stability and throughput — wasting inference optimizations built for other vendors/generations, limiting usable capacity, and costing too much.
- Their approach: trained a model on agentic coding tasks using fully asynchronous RL with mixed-precision, mixed-accelerator samplers.
- Result: with off-policy correction, training stayed stable through convergence, on par with homogeneous samplers, at lower cost.
The thread argues RL becomes much more flexible when rollout compute can be sourced elastically across hardware.
Related event: Modal's heterogeneous async RL training cuts costs while staying stable(3 posts)→
More from Infra
- Strata Runs 100 tks in Agentic Workloads but Shows Looping and Port-Conflict Issues — GodComplecs · 2026-10-14
- Ben Lorica: local AI is a routing problem, not a privacy feature — bigdata · 2026-10-14
- Meta CFO warned Zuckerberg the superintelligence bet could end 'very, very bad' — robleclerc · 2026-10-14
- Arm AGI CPU posts impressive early scores in Signal65 controlled tests — ryanshrout · 2026-10-14
- NVIDIA's upgraded DLSS 5 neural rendering model is 2x faster, up to 1.4x framerates — ctnzr · 2026-10-14
- Slide reveals NVIDIA's approved power supply vendors: Delta and LiteOn make the list — firstadopter · 2026-10-14