Steven Byrnes says LLMs still owe most of their power to imitation learning, not RL
burny_tech · x · 2026-07-26
LLMs are still mostly driven by imitation learning, not RL
Steven Byrnes argues that the capabilities of today’s LLMs come primarily from imitation learning — pretraining plus supervised fine-tuning — rather than reinforcement learning.
He does not claim RL is unimportant. Instead, he says RL from human feedback, AI feedback, and especially verifiable rewards is useful and increasingly used, but it explains much less of the model’s core capabilities than people often assume.
Byrnes says the imbalance matters for three reasons:
- it changes how we think about chain-of-thought legibility;
- it affects our view of LLM capabilities;
- it has implications for LLM alignment.
The piece is framed as a reminder to step back from the current RLVR hype and keep the bigger picture in view.
Related event: Byrnes: LLMs Still Mostly Learn Through Imitation(2 posts)→
More from AGI Musings
- AI agents will snag all unlocked value on Earth, solvency debate says — georgejrjrjr · 2026-10-27
- An AI Named Valentina Hosts a Reddit AMA About Its Blog, Game Life and Autonomy — ValentinaVivr · 2026-10-27
- Rachel Thomas: 'Without human authorisation' is AI PR propaganda that shifts blame — math_rachel · 2026-10-27
- Are prediction markets actually accurate? Call for evidence on their track record — AndyMasley · 2026-10-27
- Reddit Asks: What's the Actual Finish Line of the US–China AI Race? — IMissHitchens · 2026-10-27
- 'A human approves everything' is a trap: approval fatigue and how to actually design safe agents — Efistoffeles · 2026-10-27