Distributed RL Post-Training Achieved on 14 Macs
erfan_mhi · reddit · 2026-07-16
Pluralis Research shared a distributed RL post-training experiment where 14 consumer-grade Macs across 4 countries handled rollout generation, while a remote B200 managed bf16 gradient updates.
Key Approaches
- Rollouts were entirely connected via standard home networks without relying on data center interconnects; one of the machines was the author's own MacBook.
- The rollout phase utilized MLX + int8 inference.
- To mitigate deviations from cross-version asynchrony, the team designed two mechanisms:
- PULSE: Transmits int8 weight deltas instead of full checkpoints, reducing typical transfer size from 9GB to roughly 82MB.
- DPPO-style probability gate: Filters out about 0.3% of tokens that drift too far from the training distribution.
Results
They evaluated Stoa on PaperSearchQA, a multi-turn biomedical retrieval task:
- cover pass@1 increased from 29% to 63%
- search rate improved from 22% to 84%
Conclusion & Next Steps
The author emphasizes that consumer hardware can shoulder the bulk of rollout costs. Combining this Stoa approach with Agoro—which previously performed pre-training on hundreds of consumer GPUs—could enable migrating larger-scale inference and training entirely onto user-owned hardware.
Related event: Pluralis Runs Distributed RL Post-Training on 14 Consumer Macs(2 posts)→
More from Infra
- China’s AI arms race is increasingly defined by chips, data centers, and open models — BenBajarin · 2026-07-22
- Agent search bottlenecks are now about variance, not raw latency — rohanpaul_ai · 2026-07-22
- Gavin Baker argues Nvidia may be one of open source AI’s biggest supporters — GavinSBaker · 2026-07-22
- AI Power Demand Exposes US Energy Gap, Urging Shift from Scarcity to Abundance — bradneuberg · 2026-07-22
- Gavin Baker says Nvidia’s $630B figure would be system revenue, not all Nvidia’s — GavinSBaker · 2026-07-22
- A Firecracker-based platform says it can host 6,000 AI agents on one 256 GB server — maritime_sh · 2026-07-22