John Langford on scale threshold effects in AI research, citing the NextLat paper
JohnCLangford · x · 2026-10-13
John Langford discusses whether small-scale experiments can validate ideas that only work at scale. From an adaptive RL background he argues scale can show step-changes in ability, making small-scale validation confounded — though "only works at larger scale" can also mean you're looking at the wrong version of an approach. He cites his new NextLat paper (arXiv:2511.05963): adding self-supervised next-latent prediction provably converges latents toward belief states, injecting a recurrent inductive bias into transformers with no architectural change, yielding gains across world modeling, reasoning, planning and language modeling even at small scale.
More from AGI Musings
- Veteran game programmer calls hand-coding era "golden age" as AI now writes most code — rickasaurus · 2026-10-13
- Mathematicians' latest problem: what if AI can do our job? NYT Daily examines — stevenstrogatz · 2026-10-13
- As AI topples math problems, mathematicians split between awe and dread — LostBetsRed · 2026-10-13
- 1 in 5 books substantially written by AI now, growing from ~0% of sales in early 2023 to ~20% — TuhinChakr · 2026-10-13
- Paper: AI-written books flood Amazon, crowd out human authors' sales — TuhinChakr · 2026-10-13
- New essay: consciousness is a journey living systems do, and AI cannot — AnnaCiaunica · 2026-10-13