Modal trains RL models on mixed accelerators with async RL, matching stability at lower cost
AAAzzam · x · 2026-10-14
Modal notes that large-model RL training typically runs on a fixed homogeneous cluster of latest-gen accelerators for numerical stability and throughput — discarding inference optimizations for other vendors/generations, limiting usable capacity, and costing more than needed.
Their approach: train a model on agentic coding tasks using fully asynchronous RL with mixed-precision, mixed-accelerator samplers. With off-policy correction, training stayed stable through convergence, on par with homogeneous samplers, at lower cost.
More from Infra
- OpenRouter's Average Token Price Nearly Halved in 2026, Yet Daily Spend Tripled — maferase · 2026-10-14
- Someone Is Running Qwen 27B at ~20 tok/s on a Jailbroken PS5's GPU — LatentSpacer · 2026-10-14
- ASE Panels Spotted at OCP Showfloor, Reportedly Destined for OpenAI's Venice Data Center — BenBajarin · 2026-10-14
- Semicon West and OCP point to next bottlenecks: high-kW liquid cooling, 800VDC, semicap — BenBajarin · 2026-10-14
- Starship launch cadence capped near 145/year at current sites, a bottleneck for orbital compute — JOBhakdi · 2026-10-14
- One Postgres cluster, many previews: Alchemy custom resources give each PR its own database — samgoodwin89 · 2026-10-14