Interpretability Researchers Clash: Is Tackling Features Before Understanding Polysemanticity Reckless?

lambdacalcc · x · 2026-10-20

• A substantive researcher debate on interpretability and reward hacking: one argues a proposed feature-based intervention would be "insanely reckless" until polysemanticity is better understood.
• davidmanheim pushes back: is it really more reckless than current reward-hacking mitigations? He flags a missing question from Nick Cammarata's recent post — whether claimed monosemantic features emerge early in training and persist through RLHF/RL.
• The answer has direct implications for Anthropic's new reward hacking avoidance method; Neel Nanda and others were tagged in on the disputed monosemanticity premise.

Original post →