Tsinghua's LongZero Boosts LLM Long-Context Reasoning With Zero External Data
Tsinghua · hf · 2026-10-29
Researchers at Tsinghua University (Zhennan Wan, Jianfei Chen) released LongZero, a method that lets LLMs self-improve long-context reasoning without any external data.
Key points
- RL for long-context reasoning needs long, diverse, verifiable training data; human annotation is costly, natural data is scarce, and existing synthesis relies on external corpora or stronger teacher models — which get harder to obtain as models improve.
- LongZero uses "RL from zero data": the policy model πθ itself synthesizes structured multi-hop and retrieval-style long-context tasks (directly or via expansion), then trains with GRPO.
Paper released Oct 21 on arXiv; model available on Hugging Face.
More from Research
- WSDM 2027 to host LongSeek, first workshop on long-horizon information seeking — mrdrozdov · 2026-10-30
- Lean 4 formalization of an asymmetric metric-tensor physics paper hits GitHub — SubtleGradient · 2026-10-30
- Is sycophantic AI inflating research publications? Two mechanisms worth questioning — KLdivergence · 2026-10-30
- Tahoe AI's Tara plug-in brings 100M-cell drug datasets to Claude Code, Codex and Cursor — anshulkundaje · 2026-10-30
- Joy dataset: 100,000 hours of unscripted two-person calls to teach voice models real timing — Scobleizer · 2026-10-30
- Exo: Berkeley's self-improving agent architecture cuts costs 31-62% vs Codex — arena · 2026-10-30