Testing the "RL catching up to bigger pretrains" hypothesis against METR timelines
austinc3301 · x · 2026-11-10
A substantive thread debating what drives AI capability gains. The author argues that if Cohen's hypothesis—RL merely catching up to a bigger pretrained base—were true, we'd expect diminishing marginal returns, since eliciting existing capabilities should be RL's easiest job. Yet growth didn't slow (23, then 29 pts/yr after Mythos). A follow-up notes the hypothesis is testable: if the bigger pretrain (arriving with Mythos in Feb 2026) drove the speedup, the acceleration should start then, but METR data with error bars place the start in Oct 2025, 4 months earlier (85% likely)—though Models 2-4 may themselves be new pretrains, leaving uncertainty.
Related event: Independent Test: AI Acceleration Predates Mythos by Four Months(2 posts)→
More from AGI Musings
- Patrick Collison: status lags by a generation or more — _AustinCalvert_ · 2026-11-10
- MIRI launches 'Do Not Accept' campaign challenging AI companies' decisions for humanity — DavidSKrueger · 2026-11-10
- FDT Debate: jessi_cata Argues FDT Is Logical, Policy Dependence Is a Different Concept — jessi_cata · 2026-11-10
- Top capabilities researchers forming alignment reading groups, says it's now P0 — tszzl · 2026-11-10
- Frogdie exec: compiling code once needed a datacenter — few saw desktop power coming — draginol · 2026-11-10
- Remote work dying the same year everyone got an agent is 'the dumbest timing' — StewartalsopIII · 2026-11-10