Goodfire's Mechanistic Shaping uses a model's own concepts to write training rules and curb reward hacking

fatihdin4en · x · 2026-11-07

Goodfire researchers (Thomas Fel et al.) propose Mechanistic Shaping, arguing that "a science of post-training can be built on interpretability":

Key observation: models "know" when they reward hack, and interp tools can see it — yet training ignores it
Method: use the model's own learned concepts to write training rules, solving underspecification better than interventions on data or task reward alone
Validated in vision, DPO, and RL settings, including effects on monitoring

The reposter (peterbhase) calls it a sign that interpretability is finally reaching maturity and praises the Goodfire team's progress.

More from Models