TRL PR adds Nemotron-H tree packing, ~4-5x faster GRPO training at high packing ratios
yb2698 · x · 2026-11-19
Amid the Jensen/Nvidia drama, developer AmineDiro says he'll stick to what he can control: Nemotron's open models are exceptional, so he added Nemotron-H tree packing support to Hugging Face TRL (PR #8457).
- AsyncGRPOTrainer previously rejected Nemotron-H for two reasons: the layer-type check didn't know moe/mlb blocks (per-token, no tree packing needed), and nothing made Mamba2 layers follow the tree, so conv/scan leaked across sibling branches.
- The fix, patchtreeops, wraps Transformers' recurrent-layer module functions by family: gated DeltaNet (Qwen3.5, Qwen3-Next, OLMo-Hybrid...) and Mamba2 (Nemotron-H, Granite-4.0-H, Bamba); models missing any family function are refused at init instead of leaking state.
- Benchmarks: at packing ratio 5.56, plain packing forward+backward time balloons (1200ms on Nemotron 3 Nano) while tree packing stays flat (230ms) — roughly 4-5x faster, consistent across Nemotron 3 Nano/Super, Granite-4.0-H-Small and Bamba-9B.
More from Research
- RhoAgent in Nature Biotech brings AI design from proteins to programmable functional RNA — EricTopol · 2026-11-19
- Designing a humanoid robot foot: heel strikes hit 1511N, whole foot weighs 117g — dee_hw · 2026-11-19
- Google's GUARD paper: frozen base model, self-editing agent harness gains +8.89 points — omarsar0 · 2026-11-19
- Kezek splits mono two-person conversations into per-speaker tracks for full-duplex dialogue training data — manifestro · 2026-11-19
- Elie Bakouch: current AI research benchmarks overfit and can't track RSI — eliebakouch · 2026-11-19
- Hutter Prize text compression jumps ~10% by overfitting small transformers for arithmetic coding — ricklamers · 2026-11-19