Randomized YaRN: training on short context with sampled positions boosts 128K reasoning
gregd_nlp · x · 2026-09-05
A study slated for EMNLP 2026 Findings finds that YaRN alone isn't enough for LLM reasoning at 128K context.
The paper proposes Randomized YaRN (RYaRN): training on short-context data while randomly sampling positions from a longer length range. Results show RYaRN improves out-of-distribution reasoning accuracy, particularly at 128K context—a context-extension trick with essentially no extra data cost.
More from Research
- Tencent paper: environment evolution generates harder RL environments without watching the agent — omarsar0 · 2026-09-05
- Microsoft's AgentScope: neuro-symbolic debugging pinpoints where AI agents failed — dair_ai · 2026-09-05
- Layer-12 subspaces of Qwen3.5-2B-Base shared by interpretability researcher — Sauers_ · 2026-09-05
- EMNLP paper: AI-generated text has a consistent stylometric fingerprint across models and domains — najoungkim · 2026-09-05
- TAOCP open problems released as a dataset to benchmark frontier models — sytelus · 2026-09-05
- Clinic-in-the-Loop: why clinical trials are the real bottleneck breaking Eroom's Law — anshulkundaje · 2026-09-05