Goodfire unveils mechanistic shaping: using interpretability as training feedback
teortaxesTex · x · 2026-11-06
GoodfireAI introduced "mechanistic shaping," a mathematically rigorous framework that turns model internals into training feedback and works across domains and modalities. Demonstrated uses: mitigating reward hacking, removing shortcuts in chest x-rays, and reducing sycophancy and harmfulness. The team calls it a big step for technical alignment—moving from trial-and-error to intentionally designed behavior—while acknowledging the risk that training against internals can make models harder to monitor.
More from Safety
- METR says Anthropic redacted most key evidence from its 100-page safety assessment report — tszzl · 2026-11-06
- Canadian grant applicants may use AI but reviewers may not, researcher says that's wrong — KordingLab · 2026-11-06
- Crash test: LangGraph, Pydantic AI and OpenAI Agents SDK all double-charge payments — KaleidoscopeNew9259 · 2026-11-06
- Zvi's AI #193: Anthropic's revised S-1 warns of imminent 'dramatically accelerated progress' — TheZvi · 2026-11-06
- CPython 0day turns LLM chat templates into code execution; backdoored HF models own inference servers — dyn___ · 2026-11-06
- The '1984 Scenario': Why Never Building Superintelligence May Be the Darkest Future — AIandDesign · 2026-11-06