Modal details heterogeneous RL stack: samplers across 6 accelerators, BF16 policy updates on B200s
ying11231 · x · 2026-10-15
- Modal describes its RL training infrastructure: Miles from Radix Ark as the RL framework, SGLang samplers, and a Megatron-LM trainer updating the policy in BF16 on B200 GPUs.
- A chart shows request share served by GPU type: B300 (NVFP4/TRT-LLM) 29%, B200 (NVFP4) 20%, H200 FP8 18%, H200 BF16 9%, H100 BF16 9%, H100 FP8 8%, RTX PRO 6000 4%, A100 3%.
- Two setups: (1) mixed samplers spanning six accelerator types and three precisions (BF16, FP8, NVFP4); (2) homogeneous samplers on B200s only, with BF16 weights and BF16 KV cache. A notable engineering route for heterogeneous compute at RL scale.
More from Infra
- CoreWeave's 2032 bond yield touches 15.1%, closes at record 14.5% as AI buildout leans on private credit — asusarla · 2026-10-15
- Three questions to ask your AI supplier: capacity ownership, financing, change notice — YvesMulkers · 2026-10-15
- BlackRock CEO Larry Fink warns halting data center construction could plunge the U.S. into recession — Polymarket · 2026-10-15
- Mine Bittensor Subnet 92 by pointing Claude Code at one link — airesearch12 · 2026-10-15
- OpenAI infra lead leaves to start up after locking in double-digit GWs of compute — DesireeCachette · 2026-10-15
- Who Sells Data directory maps 159 AI training data vendors, compares Scale vs Mercor — cneuralnetwork · 2026-10-15