Goodfire Unveils "Mechanistic Shaping": Turning Model Internals into Training Feedback
mathildepapillo · x · 2026-11-06
Goodfire introduces mechanistic shaping, a single mathematically rigorous framework that transforms model internals into training feedback, working across domains, problems, and modalities.
Applied so far, it has mitigated reward hacking, removed shortcut learning in chest x-ray classifiers, and reduced sycophancy and harmfulness. The team calls it a significant step forward for technical alignment.
They also flag a known risk: training against a model's internals can make it harder to monitor—an issue they explore alongside the technique itself.
More from Safety
- Miles Brundage cites METR: Anthropic will pack 4 months of progress into 2 before Congress — nitarshan · 2026-11-06
- Reporter's ChatGPT 'sharpening' of news photo reignites debate over AI edits altering meaning — haydenfield · 2026-11-06
- Ajeya Cotra: redacted evidence undermines AI risk verification and safety cases — ajeya_cotra · 2026-11-06
- Vinod Khosla: AI plus quantum computers will break most encryption before 2030 — rohanpaul_ai · 2026-11-06
- Yale study: ChatGPT recommended a Senate candidate in 79% of political chats — luisdans · 2026-11-06
- Compound Security: open-source skill for agent-driven vulnerability patching — hamostaf04 · 2026-11-06