Tree packing lands in TRL AsyncGRPO, cutting agentic RL trainer compute 26x

SergioPaniego · x · 2026-10-15

• Tree packing has merged into AsyncGRPO in Hugging Face TRL: GRPO forks G rollouts from the same prompt, and these multi-turn rollouts share most of their history. Unlike sequence packing, which re-forwards shared tokens per sample, tree packing forwards each unique token only once.
• The shipped version is a single FA3 varlen call (replacing an earlier FlexAttention implementation), keeping each token's own target, advantage, and previous log-prob. It was tested against sequence packing on dense, sliding-window, softcap, MoE, and VLM models, costing 1% when nothing can be packed.
• On a real agentic RL run on MiMo Envs, tree packing reached the same reward with 26x less trainer compute.

Original post →

More from Infra

• patio11 on multi-agent dev: clearly the future, and we'll buy so much inference — IgorCarron · 2026-10-15
• Hugging Face Hub makes model tensor inspection up to 20x faster, DeepSeek-V3.2 drops from 9.2s to 0.45s — victormustar · 2026-10-15
• RTX 5060 Ti 16GB surges up to 144% in Japan, hitting $1,292 as RTX 5080 tops $2,400 — chemist_slime · 2026-10-15
• Fish Audio Drama 3 lands in vLLM-Omni: 128 concurrent streams on one H200 — vllm_project · 2026-10-15
• 1,174 Macs form Darkbloom inference network, growing 4.5x in 60 days at half the cost — gajesh · 2026-10-15
• ByteDance in advanced talks with MediaTek for custom AI data-center chips, Bloomberg reports — mark_k · 2026-10-15

Infra channel →