White Circle open-sources Halo, a distributed post-training framework hitting 2.8x TRL throughput

rohanpaul_ai · x · 2026-09-22

White Circle unveiled Halo, an open-source distributed post-training framework positioned above HuggingFace TRL but below the full Megatron stack.

On 8×B300 GPUs, Halo delivers up to 2.8x the training throughput of stock TRL; with both sides sharding ZeRO-3 it's 2.7x while using 25% less peak memory, with larger margins over other benchmarked frameworks
Rather than reimplementing architectures, Halo wraps existing Transformers models with expert, context, tensor, and expert-tensor parallelism
Supports multi-turn RL directly on HuggingFace-format models, scaling from a single GPU to a cluster

More from coding & agent