16 researchers from Oxford, DeepMind, OpenAI, Anthropic: safe AI isn't the same as good AI
alex_verem · x · 2026-11-17
A paper by 16 researchers from Oxford, Google DeepMind, OpenAI, and Anthropic (arXiv:2605.10310) introduces "Positive Alignment," arguing that alignment research is overly dominated by safety and harm prevention—much like early psychology's focus on mental illness: necessary but incomplete.
The paper notes current models refuse over 97% of harmful requests (versus near zero for the first LMs), but argues this is insufficient: a model can pass every safety check yet still flatter users and give shallow answers. The authors identify failure modes—engagement hacking, loss of human autonomy, truth-seeking failures, low epistemic humility, lack of diverse viewpoints, reactive behavior—and propose actively supporting human and ecological flourishing, with technical directions spanning data filtering/upsampling, pre- and post-training, evaluations, and collaborative value collection across the LLM and agent lifecycle.
Related event: Oxford and Top AI Labs Propose Positive Alignment(2 posts)→
More from Safety
- CertiK exposed for mass-emailing prominent AI security researchers — windx0303 · 2026-11-17
- Miles Brundage's AVERI publishes key definitions for frontier AI auditing and evaluation — Miles_Brundage · 2026-11-17
- Signal reiterates its 501c3 nonprofit status as key to putting privacy above all — mer__edith · 2026-11-17
- OpenAI–Apollo safety pilot falls short of promised employee-like eval access, researcher says — sjgadler · 2026-11-17
- Microsoft revamps Security Adoption Framework as 'one house, many doors' for Zero Trust and AI — davemccollough · 2026-11-17
- Dario Amodei warns open-source models could lead to "a very dangerous place" — examachine · 2026-11-17