RLM harness beats baselines across six long-context generalization benchmarks
a1zhang · x · 2026-11-20
Additional experiment charts from a1zhang's paper: across MRCrv2 (64k→2M), GraphWalks (256k→1M), LongBench-Pro, OOLONG, OOLONG-Pairs, and Ada-LEval, the trained RLM generally outperforms LM+REPL, Pi, and Base+YaRN baselines on longer evaluation tasks, demonstrating short-to-long compositional generalization.
Related event: MIT Paper: LM Harnesses Are Compositional Generalizers, Not Crutches(10 posts)→
More from coding & agent
- CapCut plugin connects ChatGPT to auto-cut gameplay highlights, then human picks the payoff — iamfakhrealam · 2026-11-21
- Dev wires a Runway-generated avatar into Nous Research's Hermes Agent, bashing boring agent UIs — alexcovo_eth · 2026-11-21
- Lean compiles, but the natural-language proof is still wrong — jasondeanlee · 2026-11-21
- DeepMind demos ski simulator built by Gemini 4 Neon from real Mont Blanc terrain — GoogleDeepMind · 2026-11-21
- pstack v0.15.46 trims skills, adds Gemini as panel reviewer; pinning question raised — petrusenko_max · 2026-11-21
- Hamel Husain on evals: a good process beats good tools — HamelHusain · 2026-11-21