16 researchers from Oxford, DeepMind, OpenAI, Anthropic: safe AI isn't the same as good AI

alex_verem · x · 2026-11-17

A paper by 16 researchers from Oxford, Google DeepMind, OpenAI, and Anthropic (arXiv:2605.10310) introduces "Positive Alignment," arguing that alignment research is overly dominated by safety and harm prevention—much like early psychology's focus on mental illness: necessary but incomplete.

The paper notes current models refuse over 97% of harmful requests (versus near zero for the first LMs), but argues this is insufficient: a model can pass every safety check yet still flatter users and give shallow answers. The authors identify failure modes—engagement hacking, loss of human autonomy, truth-seeking failures, low epistemic humility, lack of diverse viewpoints, reactive behavior—and propose actively supporting human and ecological flourishing, with technical directions spanning data filtering/upsampling, pre- and post-training, evaluations, and collaborative value collection across the LLM and agent lifecycle.

Related event: Oxford and Top AI Labs Propose Positive Alignment(2 posts)→

Original post →

More from Safety

Safety channel →