LongStraw: Million-Token Long-Context RL
mindlab-research · hf · 2026-07-17
LongStraw proposes an execution stack tailored for million-token level RL post-training, aiming to perform long-context reinforcement learning within a fixed GPU budget.
The core idea is to perform an autograd-free evaluation of the shared prompt first, retaining only the model states required for subsequent tokens. Short response branches are then replayed sequentially, compressing the training graph and peak VRAM usage at the cost of increased replay time. The authors implemented this using Qwen3.6-27B (featuring hybrid recurrent and full attention) and GLM-5.2 (with compressed attention MoE).
Experimental results include:
- On 8 H20 GPUs, Qwen's grouped scoring and response backward can handle 2.1M positions; increasing the group size from 2 to 8 only raises peak VRAM by 0.21 GB.
- Standalone stress testing reached 4.46M positions.
- On 32 H20 GPUs, they validated an end-to-end execution path for GLM-5.2 using a 2.1M-token prompt spanning 78 layers.
The authors emphasize that these experiments primarily demonstrate execution capability rather than full training correctness, as the captured prompt state is detached, and some distributed forward/gradient combination paths are not yet complete.
More from Infra
- Engram's random reads don't suit SSDs; CPU-memory over NVLink could serve all 72 GPUs — bookwormengr · 2026-09-11
- 80% of the DIY LLM inference hype posters have already quit — it's brutally hard systems work — abhijithneil · 2026-09-11
- Hugging Face's Ultra Scale Playbook: a free book on training LLMs on GPU clusters — mdancho84 · 2026-09-11
- Is inference latency becoming the biggest bottleneck for production AI agents? — Euphoric_Sea632 · 2026-09-11
- LLM Serving Metrics Thread: Why TPOT and Uptime Make or Break User Experience — abhijithneil · 2026-09-11
- PlanetScale launches sharded Postgres: 768 servers acting as one, 1PB scale — dhruv2038 · 2026-09-11