LongStraw: Million-Token Long-Context RL
mindlab-research · hf · 2026-07-17
LongStraw proposes an execution stack tailored for million-token level RL post-training, aiming to perform long-context reinforcement learning within a fixed GPU budget.
The core idea is to perform an autograd-free evaluation of the shared prompt first, retaining only the model states required for subsequent tokens. Short response branches are then replayed sequentially, compressing the training graph and peak VRAM usage at the cost of increased replay time. The authors implemented this using Qwen3.6-27B (featuring hybrid recurrent and full attention) and GLM-5.2 (with compressed attention MoE).
Experimental results include:
- On 8 H20 GPUs, Qwen's grouped scoring and response backward can handle 2.1M positions; increasing the group size from 2 to 8 only raises peak VRAM by 0.21 GB.
- Standalone stress testing reached 4.46M positions.
- On 32 H20 GPUs, they validated an end-to-end execution path for GLM-5.2 using a 2.1M-token prompt spanning 78 layers.
The authors emphasize that these experiments primarily demonstrate execution capability rather than full training correctness, as the captured prompt state is detached, and some distributed forward/gradient combination paths are not yet complete.
More from Infra
- Nearly half of Claude revenue now flows through cloud marketplace commitments, up from 11% — BenBajarin · 2026-10-26
- Anthropic ARR goes vertical after price cuts: tokens up 4x, $100B+ ARR back in play — GavinSBaker · 2026-10-26
- Alex Banks argues power caps GPU supply, real risk is demand cooling first — thealexbanks · 2026-10-26
- Analyst: Ant has the most fungible AI workload of any model provider today — BenBajarin · 2026-10-26
- Anthropic S-1: training infra edge cut per-task cost 40% and doubled revenue growth — BenBajarin · 2026-10-26
- Rewriting scaling law study: optimal synthetic data share rises from 10% to 32% with compute — heghbalz · 2026-10-26