Anthropic: model activations predict ripple effects of value alignment training across 66 values
dhadfieldmenell · x · 2026-10-15
Anthropic published "Predicting Alignment Generalization with Value Representations," asking whether downstream effects of value alignment can be predicted in advance.
Key findings:
- The team trained models on 66 different values and measured ripple effects on other values when training on any one of them.
- Model internal activations proved far more predictive of these generalization effects than textual value descriptions.
- The work is a step toward predicting the side effects of alignment training and understanding how post-training shifts model behavior in unexpected ways.
More from Safety
- Korea clears 20 humanoid robots to work on real factory floors — emmanuelvivier · 2026-10-15
- Microsoft locks down Copilot against prompt injection with deterministic isolation layer — emmanuelvivier · 2026-10-15
- Gladstone AI report: labs automate R&D pipelines, RSI could start in 3-12 months — sourdub · 2026-10-15
- Hackathon team claims it found a large fleet of agents from a non-US frontier lab — NathanpmYoung · 2026-10-15
- Argon hits cyber, CBRN CCLs and misalignment TCL; report flags CoT transparency — vkrakovna · 2026-10-15
- AI Doesn't Care If It's Wrong — You'll Face the Court Consequences — adariostrange · 2026-10-15