Safety researchers blast OpenAI/Anthropic plan: automating alignment research with possibly misaligned AI
trevposts · x · 2026-11-12
Nathan Calvin says that as far as he knows, the plan for aligning superhuman AI at OpenAI, Anthropic and other labs is essentially to automate AI safety research using AIs that are increasingly hard to monitor and may themselves be misaligned — with no more sophisticated backup plan, something the public and governments would find alarming.
Another safety researcher goes further, calling the plan "really, really bad": labs have a moral duty to exit any commercial race toward vastly superhuman AI, inform the public of imposed risks, and allow unbiased risk assessment — yet are instead lobbying against the best shot at regulation, obscuring their own risk views, and letting only a tiny number of biased external evaluators see the needed information.
More from AGI Musings
- Most biology AI models learn only one layer, and the big bet hinges on mapping — anshulkundaje · 2026-11-12
- 'This was AFTER GPT-3': why failing to see new things makes you useless at prediction — repligate · 2026-11-12
- Congressional AI bill fight: federal preemption as 'floor, not a ceiling' — ShakeelHashim · 2026-11-12
- Chalk mark worth $1,000: why impressive-looking long reports fool people — JacquesThibs · 2026-11-12
- repligate blasts bio-anchors report: "outside view" forecasting is dead wrong — JacquesThibs · 2026-11-12
- Data flywheel is fake: only deployment data where models fail is valuable, argues researcher — chris_j_paxton · 2026-11-12