Mechanistic interpretability will never truly be solved, researcher argues
burny_tech · x · 2026-10-25
AestherML argues mechanistic interpretability will never truly be solved: humans hold 3 concepts in working memory while models like Mythos contain trillions, some concepts map to abstractions we can't understand, and any single change ripples unpredictably — with continual learning making it worse. Alignment resources, they contend, should go to getting rewards right rather than post-hoc ablation and SAE management.
More from AGI Musings
- Blogger claims hyperscaler-backed AI regulation aims to crush small labs, entrench monopoly — Ghost_Pilot_MD · 2026-10-25
- Toddler development as a metaphor: evolutionary pre-training and efficient post-training — paigeinsf · 2026-10-25
- "I do miss programming": dev says AI coding killed the flow state — generativist · 2026-10-25
- Anthropic CEO on learning to code: programming goes first, but your 5% still matters — 2C_ornot2C · 2026-10-25
- After 22 years studying workforces: the AI question isn't replacement, it's 'what kind of people' — 2C_ornot2C · 2026-10-25
- AI-generated physics ideas mostly already done in past 2 months, physicist finds — burny_tech · 2026-10-25