Pretraining vs RL debate is silly — evidence says you need both, argues voooooogel

voooooogel · x · 2026-11-21

Reacting to the "is pretraining or RL more important" debate, voooooogel calls it a chicken-and-egg argument. His case: we have good evidence pretraining alone isn't enough (405B base hasn't proved anything), and very strong evidence that pretraining + RL works. If someone wants to claim LLMs are hitting a wall because the pretraining base or "convex hull of human ideas" is exhausted, they should say so directly — phrased that way, it sounds like plenty of bets that didn't age well. The question of which component is "more important" is moot; whether pretraining gains have truly plateaued is the real debate.

Original post →

More from AGI Musings

AGI Musings channel →