Goodfire unveils mechanistic shaping: using interpretability as training feedback
akbirthko · x · 2026-11-06
Interpretability startup Goodfire introduces mechanistic shaping, a single mathematically rigorous framework that turns model internals into training feedback across domains and modalities. They've used it to mitigate reward hacking, remove shortcuts in chest x-rays, and reduce sycophancy and harmfulness, calling it a step forward for technical alignment — while flagging the risk that training against internals can make models harder to monitor.
More from Safety
- UK cyber bill adds 'loss of oversight or control of an AI agent' clause, still undefined — mclynd · 2026-11-06
- Dev warns: don't link your bank account to AI companies — arthurcolle · 2026-11-06
- Debate: do confidentiality deals justify staying silent on AI timelines? — albrgr · 2026-11-06
- Microsoft: threat actors use agentic AI tool ARTEX to harvest exposed secrets from GitHub — davemccollough · 2026-11-06
- Researchers clash over Anthropic report: conclusions rest on subjective staff assessments — danielrock · 2026-11-06
- Dimitris Papail: safety researcher may be the last technical AI job left — DimitrisPapail · 2026-11-06