Yudkowsky Mocks Interpretability Research: Finding 'Eiffel Tower Neurons' Isn't Alignment
nptacek · x · 2026-10-16
A quoted thread captures Yudkowsky's critique of mechanistic interpretability: he argues real alignment requires predicting a model's thinking in enough detail to anticipate deviations from human thought, and that SFT/RL are just hammering from the outside. Seeing researchers celebrate discovering neurons that light up for the Eiffel Tower, he sarcastically calls it a bare prerequisite for the important part 'eventually, someday.' nptacek's repost highlights the enduring rift between alignment purists and the interpretability research community.
More from AGI Musings
- Blogger predicts recursive self-improvement arrives around November 2027 — imjustnewatai · 2026-10-16
- 31 Tech Curves Visualized: Kurzweil's $1,000-Brain-Compute Call Missed by 400x — cpaik · 2026-10-16
- DeepMind Paper: First-Price Auctions in Agentic Markets Reach 1-1/e^2 Efficiency — dair_ai · 2026-10-16
- Garrison Lovely's new book 'Obsolete' argues AGI is neither impossible nor inevitable — GarrisonLovely · 2026-10-16
- Majmudar clarifies: controllability may be solved in 1-2 years, not full alignment — MajmudarAdam · 2026-10-16
- AI-generated fake Feynman lectures are flooding the internet, and they're bad — dreamwieber · 2026-10-16