Researchers Debate Where Frontier Models' Reasoning Comes From

Starting September 25, AI researcher Yoav Goldberg and several colleagues debated on X where frontier LLMs' reasoning ability comes from. Goldberg expressed an industry-level puzzle: current reasoning traces are so good he cannot build a mental model of how they emerge from RL training even with initial CoT ability and billions of rollouts. Barak Rotblat shared the confusion. The next day, former OpenAI researcher Lukas Kaiser offered an explanation.

Confirmed

• Goldberg clarified he questions the specific type and level of reasoning in frontier models, not reasoning in general, and he explicitly does not consider these abilities emergent.
• Goldberg noted the public tech stack roughly explains DeepSeek R1-era "short and okay" reasoning, but what fills the gap to today's long, complex reasoning is unexplainable to him.
• Goldberg framed it as asking "what is a chess master's training routine" rather than "how does the human brain play chess"—the answer likely lies in concrete training procedures, not a mysterious latent process.
• Kaiser responded that the pipeline is largely known: take massive amounts of model reasoning traces, have another model clean and summarize them (with heavy human prompting, domain knowledge, and filtering), then continue RL on this synthetic data.
• On RL generalization, Kaiser said the key is finding enough diverse high-quality tasks, and suggested arXiv lemma tasks could test whether strong reasoning reflects targeted RL or true generalization.

Unconfirmed

• Whether reasoning comes mainly from RL emergence or large-scale SFT annotation remains uncertain even to Goldberg; brianryhuang's claim that superhuman reasoning is theoretically achievable with zero SFT is unproven empirically, and Kaiser's account remains his personal explanation.

Why it matters

• The debate touches the core mechanism question of the reasoning-model boom, directly bearing on training recipes, data investment, and reproducibility. Kaiser's account suggests large-scale cleaned synthetic reasoning data plus RL may be the bridge from R1-style short reasoning to today's long chains, while the arXiv lemma idea offers an operational test of generalization versus targeted training.

2026-09-25 ~ 2026-09-26 · 12 related posts

• [source] Yoav Goldberg: LLM reasoning traces are 'too good' — unclear how they emerge from RL — yoavgo · 2026-09-25
• Yoav Goldberg: I can't explain how today's long LLM reasoning traces emerged — odedbendov · 2026-09-25
• What bridged the gap from R1-style short reasoning to today's long chains? — yoavgo · 2026-09-25
• [source] Zero SFT needed: researcher argues superhuman reasoning emerges from pretraining plus scaled RL — brianryhuang · 2026-09-25
• "What's the Secret Sauce?" Barak Rotblat on How RL Yields Elite LLM Reasoning — BarakRotblat · 2026-09-25
• Yoav Goldberg: Frontier Model Reasoning Is Not "Emergent" — yoavgo · 2026-09-25
• Yoav Goldberg: I can't explain why SOTA reasoning traces are this good — LabYosi · 2026-09-25
• Yoav Goldberg: Frontier Model Reasoning Looks Like a Training Process Question, Not a Black Box — yoavgo · 2026-09-25
• [source] Lukas Kaiser explains the recipe: distilled reasoning traces plus RL — lukaszkaiser · 2026-09-26
• Ex-OpenAI's Kaiser: reasoning models trained on cleaned model traces plus RL — yoavgo · 2026-09-26
• OpenAI's Łukasz Kaiser on RL generalization: agent reasoning may generalize more than expected — lukaszkaiser · 2026-09-26

1 near-duplicate retellings: brianryhuang