Researchers Debate: Why RL Training on Agent Harnesses Generalizes Better
srush_nlp · x · 2026-07-23
A discussion sparked by Sholto Douglas and Rachel Lim's research. Sasha Rush questioned the experiment for merely comparing a raw LLM to one with a Python interpreter, failing to see how it proves the need for token-by-token input. Sholto clarified that the experiment compares direct base LM training against training an LM equipped with a harness, with evaluations deliberately introducing out-of-distribution task lengths or topics. Data shows that RL training on the harness yields better generalization, an effect not seen when arbitrarily applying standard harnesses. He emphasized that the token-by-token input is crucial because the harness shapes the context trajectories for the underlying LM's individual calls.
Related event: LLM Generalization Debate: Intrinsic Model or Harness Contribution?(11 posts)→
More from Research
- Penn's Kording Lab opens nominations for Spring 2027 speaker series — KordingLab · 2026-10-27
- neuron-explorer adds MLX backend for per-neuron steering on Apple Silicon — arthurcolle · 2026-10-27
- OpenAI says its models produced a Yang-Mills existence proof, formalized in Lean — exophades · 2026-10-27
- Bubeck hails subexponential-time 3SAT algorithm as biggest TCS surprise in 20 years — danintheory · 2026-10-27
- Greg Brockman announces AI-assisted discoveries in math, CS, and quantum field theory — gdb · 2026-10-27
- AI is simultaneously flooding journals with slop and cleaning up human slop — _akpiper · 2026-10-27