HSDP cuts async GRPO step time 40% and boosts tokens/s 1.67x vs FSDP on Qwen3-30B

FerdinandMom · x · 2026-11-18

A hands-on benchmark of async GRPO + LoRA on Qwen3-30B-A3B via Hugging Face Jobs (TRL AsyncGRPOTrainer, 3 vLLM rollout Jobs, trainer on 4x H200 across two Jobs) shows switching FSDP to HSDP cuts time per step by 40% and raises throughput 1.67x.

Related event: HSDP Beats FSDP by 40% in Async GRPO Training on HuggingFace Jobs(3 posts)→

Original post →

More from Infra

Infra channel →