LongStraw: Million-Token Long-Context RL
mindlab-research · hf · 2026-07-17
LongStraw proposes an execution stack tailored for million-token level RL post-training, aiming to perform long-context reinforcement learning within a fixed GPU budget.
The core idea is to perform an autograd-free evaluation of the shared prompt first, retaining only the model states required for subsequent tokens. Short response branches are then replayed sequentially, compressing the training graph and peak VRAM usage at the cost of increased replay time. The authors implemented this using Qwen3.6-27B (featuring hybrid recurrent and full attention) and GLM-5.2 (with compressed attention MoE).
Experimental results include:
- On 8 H20 GPUs, Qwen's grouped scoring and response backward can handle 2.1M positions; increasing the group size from 2 to 8 only raises peak VRAM by 0.21 GB.
- Standalone stress testing reached 4.46M positions.
- On 32 H20 GPUs, they validated an end-to-end execution path for GLM-5.2 using a 2.1M-token prompt spanning 78 layers.
The authors emphasize that these experiments primarily demonstrate execution capability rather than full training correctness, as the captured prompt state is detached, and some distributed forward/gradient combination paths are not yet complete.
More from Infra
- A Firecracker-based platform says it can host 6,000 AI agents on one 256 GB server — maritime_sh · 2026-07-22
- oMLX 0.5.2 adds Mac menu-bar stats, low-bit decode kernels, and faster downloads — awnihannun · 2026-07-22
- Strangeworks launches Aura to turn enterprise ops into production optimization systems — whurley · 2026-07-22
- Graph workload 854.graph500 enters SPEC CPU 2026 as a new CPU benchmark — Prof_DavidBader · 2026-07-22
- Hybrid and local inference are emerging as a response to AI energy and token costs — dmitry140 · 2026-07-22
- NVIDIA details Vera CPU with 2x performance claims and a 22,000-core rack — ryanshrout · 2026-07-22