Goodfire Proposes Mechanistic Shaping: Turning Model Internals into Training Feedback

On November 6, interpretability company Goodfire (Thomas Fel et al.) released a new framework called "mechanistic shaping," arguing that "post-training science can be built on top of interpretability": models actually "know" when they are reward hacking, and this signal is observable—but existing training pipelines ignore it.

Confirmed

• It is a mathematically rigorous unified framework that turns a model's internal representations/features into training feedback signals, directly controlling which concepts the model relies on during training.
• The framework claims to work across domains, tasks, and modalities.
• Verified applications demonstrated in the paper include: mitigating reward hacking, and removing/eliminating shortcut-learning features in chest X-ray classifiers (the figure shows an example of feature annotation on chest X-rays); training rules can even be written using concepts the model itself has learned, customizing the basis for its decisions.
• The method has been validated in the vision domain.

Why it matters

Multiple commenters (@teortaxesTex, @fatihdin4en, @mathildepapillo, etc.) see it as a major step for "technical alignment": it upgrades interpretability research from a "post-hoc analysis tool" to a "training feedback signal," providing an actionable training-layer approach to alignment challenges such as suppressing reward hacking.

All information in this cluster comes from reposts; no first-hand post from the original publisher was seen. The above is a merged paraphrase of multiple reposted contents.

2026-11-06 ~ 2026-11-06 · 5 related posts

• Mechanistic Shaping uses model's own concepts to write training rules — fatihdin4en · 2026-11-06
• Goodfire unveils mechanistic shaping: using interpretability as training feedback — teortaxesTex · 2026-11-06

3 near-duplicate retellings: mathildepapillo · mathildepapillo · mathildepapillo