Deep dive: running RL on heterogeneous hardware — Blackwells for training, TPUs for inference
A_K_Nain · x · 2026-10-22
- 作者梳理了在异构加速器上做 LLM RL 后训练所需的系统设计:用 Blackwell 做 trainer、TPU 做推理采样,并引用 Seer、RollPacker、Jet-RL 等工作指出 rollouts 占 RL 墙钟预算的 65-90%。
- 关键张力在于训练-推理数值失配:这类失配即使同一 GPU 上不同 prefill/decode kernel 也会出现,跨硬件(如 AReal-Hex 用 H20 做采样,取其 4TB/s 带宽、仅 15% H800 BF16 算力)会进一步放大,可能直接导致 RL 崩溃。
- 文章讨论了 Routing Replay、Score Centering、Masked Importance Sampling 等业内已有思路如何迁移到异构系统。作者坦承暂无自己的实验结果,属于基于既有研究与系统细节的外推分析,欢迎反馈。
More from Infra
- Omarchy package server is timing out: updates failing on cursor, grok-bot, codex-desktop packages — mark_k · 2026-10-22
- DeepSeek V4.1 Flash hits 1,521 tok/s on Fireworks Ultrafast, ships Tetris in 11 seconds — nicolechirps · 2026-10-22
- Citi projects Big Four hyperscaler capex to hit $1.62T annually by 2030, $5.35T cumulative — Beth_Kindig · 2026-10-22
- NVIDIA open-sources SimReady Central: 2,000+ sim-ready 3D assets on Hugging Face — lhoestq · 2026-10-22
- From devrel to CTO: juggling 12 cloud and AI providers and still running out of compute — NERDDISCO · 2026-10-22
- Coinbase's x402 was built to kill API keys; Stripe ships Machine Payments SDK for agent payments — kleffew94 · 2026-10-22