One-line HSDP tweak speeds up cross-node GRPO training by 40% on HuggingFace Jobs

Blogger FerdinandMom ran TRL's AsyncGRPOTrainer with LoRA on HuggingFace Jobs to train Qwen3-30B-A3B (3 vLLM Jobs generating rollouts, 2 trainer Jobs training across nodes), and found that a single code change cut per-step training time by 40% and boosted token throughput 1.67x — the key was rethinking the communication topology rather than the algorithm itself.

Confirmed

Why it matters

2026-11-18 ~ 2026-11-18 · 5 related posts

Primary sources