Blog argues LLM capabilities still come mostly from imitation, not RLVR
burny_tech · x · 2026-07-25
LLMs still seem to be powered mostly by imitation, not reinforcement
The post argues that the main source of today’s LLM capabilities is still imitative learning — pretraining plus SFT — rather than reinforcement methods such as RLAIF or RLVR.
Core claim
The author steps back from the current excitement around RLVR and says that when you look at a final trained model, most of its broad capability appears to come from imitation, with reinforcement playing a smaller role.
Why it matters
The essay says this framing changes how we should think about:
- where LLM capabilities actually come from
- how training effort should be allocated
- what kinds of future improvements are likely to matter most
Related event: Byrnes: LLMs Still Mostly Learn Through Imitation(2 posts)→
More from Research
- New review: how prefrontal cortex dynamics turn world models into flexible behavior — AnnaCiaunica · 2026-10-27
- LIBERO-MAX: Robot Policies Drop 11–26 Points When the World Changes Mid-Task — drmapavone · 2026-10-27
- Kolab Benchmark Uses Pro Screen Recordings to Show AI Agents Overestimate Collaboration Quality — minnesotanlp · 2026-10-27
- One BBQ Example Pushes LLM Fairness Scores Past 90%, EMNLP Paper Questions Benchmarks — radamihalcea · 2026-10-27
- Real2Sem2Real turns one phone video into 2,000+ robot demos, matching 150 teleop demos — Scobleizer · 2026-10-27
- Understanding MuonH (Hyperball) through weight norm geometry of scale-invariant networks — jasondeanlee · 2026-10-27