RL on heterogeneous hardware: one B300 trainer, four chips, plain HTTP, zero restarts
DavidBennett__ · x · 2026-11-16
ai& Research published a field guide to running RL on heterogeneous hardware, presented at the SGL Summit. Setup: a two-node B300 trainer consumed rollouts from B300, H200, MI355X and MI300X engines across two vendors over plain HTTP for ten hours and ten RL steps, with every adapter sync landing on all engines and zero restarts.
Key findings:
- The wire barely matters with LoRA: InfiniBand syncs 6.5× faster than HTTP but saves under 2% of a step, since only a 4 GB adapter moves, not the 709 GB base model.
- More rollout nodes help until the trainer is the limit: adding a second MI355X node nearly doubled rollout throughput (1.9×), making the run trainer-bound.
- Software is the hard part: identical launch configs landed on different engines per chip; most failures involved rarely-run LoRA paths.
Measured throughput: MI355X 2,490 tok/s, B300 1,790 tok/s, MI300X 1,180 tok/s, H200 1,030 tok/s. Takeaway: train on one chip, roll out on any — the bottleneck is software adaptation, not hardware or networking.
More from Infra
- Thinking Machines releases Inkling 2 Small; 4-GPU setup hits 380K KV cache concurrency — LiTianleli · 2026-11-16
- Samsung, Micron, SK hynix face RAM antitrust suit over 697% price surge in 18 months — wen_ragnarok · 2026-11-16
- Cynapse: a free open-source tool to fact-check and audit your local LLMs — _AustinCalvert_ · 2026-11-16
- WEKA open-sources kvlab: replay real agent traffic to benchmark KV cache offload — AccBalanced · 2026-11-16
- A shortlist of LLM inference throughput benchmarks: AIPerf, AgentX, llm-inference-bench — AccBalanced · 2026-11-16
- KubeCon's big theme is inference: a full-stack breakdown from DRA GPU slicing to vLLM and llm-d — AccBalanced · 2026-11-16