DeepMind's Jacob Hilton: interpretability lags far behind model scaling
JacobHHilton · x · 2026-10-16
In a discussion of his interpretability blog post, DeepMind researcher Jacob Hilton argues that bigger models develop crisper features and finer conceptual distinctions, but at a far slower rate than the models themselves grow—the remaining >99.99% is likely still as messy as small models. He also says a colleague's public optimism paints an unrealistic picture of the field's state, adding that neither of them is likely to succeed on the timelines they'd hope for.
Related event: ARC's Jacob Hilton Sparks Debate: Interpretability Far From Solved(22 posts)→
More from AGI Musings
- Reddit debate: Are AI services subsidized like early Uber — and will prices spike post-IPO? — ionicfellow · 2026-10-17
- Economists warn AI boom is a black box of interlocking bubbles, not a normal cycle — PAstynome · 2026-10-17
- Google grounds AI agents in wet-lab robotics, discovers nanomolar cancer protein binders — davidpattersonx · 2026-10-17
- TikTok reshaped everyday language — copywriting may be the last marketing skill standing — mallow610 · 2026-10-17
- Will Bryk's example: whether a model may say a word could depend on the user — willcb · 2026-10-17
- AI welfare researcher: human-shaped labels like 'slavery' are anti-useful — AaronBergman18 · 2026-10-17