Interpretability debate: are smaller models harder to understand?

giosullutrone99 · x · 2026-10-16

This reply prompts Jacob Hilton's candid take on interpretability's real state. The questioner, partway through Hilton's blog post, asks whether it matches the claim that smaller models have coarser features and are therefore harder to understand. Hilton responds that bigger models grow crisper features but far slower than they scale, and warns that public optimism in the field is unrealistic.

Original post →