A missing parameter in scaling laws: H, the human effort behind learning signals
lateinteraction · x · 2026-10-16
Researcher shikibmehri argues scaling laws are missing a parameter H — the human effort/time leveraged to produce learning signal — and that most recent wins come from scaling H rather than N or D: ChatGPT beat OPT via heavy investment in instruction tuning and RLHF data; coding/math gains come from human demonstration data and process rewards; distillation lets competitors scale D while free-riding on your scaled H; data vendors profit because scaling H costs far more than D/N; and the highest-signal human effort delivers near-infinite returns. An accompanying chart contrasts Kaplan and Chinchilla scaling laws, which contain only N and D.
More from AGI Musings
- Duke Study of 4,439 People Finds Coworkers Quietly Rate AI Users as Lazier — aakashgupta · 2026-10-16
- DSPy Author's $70 O'Reilly Book Rejected by Engineer Who Now Just Tells Claude What to Do — lateinteraction · 2026-10-16
- 'A machine proving new math, a car driving up a wall': AI researcher on how weird it's getting — rand_longevity · 2026-10-16
- Yacine on training: 'Neural nets are lazy stupid idiots — I build hells to make them ubermensch' — yacineMTB · 2026-10-16
- Jeff Clune: Moving the Goalposts Will Be Humanity's Last Job — jeffclune · 2026-10-16
- Replit CEO Amjad Masad Says the Future of Work Is the 'Self-Driving Company' — TechCrunch AI · 2026-10-16