Do features get crisper with scale? It's only gradual, argues Jacob Hilton
JacobHHilton · x · 2026-10-17
Responding to claims that features and circuits get crisper with scale, Jacob Hilton cites Gurnee & Tegmark's "Language Models Represent Space and Time" (arXiv:2310.02207), which shows linear spatial and temporal representations improving across Llama-2 model sizes (Figure 2).
But he argues this is only gradual improvement against exponentially growing model size — paired with his earlier point that an 88-node addition circuit scores 50% on faithfulness, the picture is that interpretability does not automatically keep pace with capability.
Related event: ARC's Jacob Hilton Sparks Debate: Interpretability Far From Solved(22 posts)→
More from Research
- Anthropic researchers: superposition isn't the blocker, interpretability is more tractable than assumed — akbirthko · 2026-10-17
- GEMMU benchmark exposes 3 pitfalls that made multimodal unlearning look better than it is — _vztu · 2026-10-17
- Parser-free code index for AI agents cuts grep bills 42% with 4x smaller tool outputs — microbass · 2026-10-17
- Quiz: how many trainable params does GPT-2 use just for embed and unembed? — Sauers_ · 2026-10-17
- Google grounds AI agents in wet-lab robotics, discovers nanomolar cancer protein binders — davidpattersonx · 2026-10-17
- Paper stuck at JMLR for over a year: researcher shares where to submit instead of TMLR — avt_im · 2026-10-17