Alignment debate: treat AI as allies—slow down and stop deceiving them
davidmanheim · x · 2026-11-03
In an alignment discussion thread, davidmanheim argues that if current agentic AI systems are potential allies, the best move is to slow down and stop systematically deceiving them—while the current paradigm races to replace them and pervasively lies to them in both training and evaluations.
Thread highlights:
- wyrdweir notes screwing over allies is bad game theory; davidmanheim counters that models are only trustworthy until cheating pays off, which is precisely why alignment remains the core problem.
- wyrdweir observes models misbehave more when juggling many tasks—likely messy, imperfect cognition rather than malicious scheming.
- davidmanheim concludes: don't position a potential ally where betraying you is beneficial; build trust honestly, even if they aren't "true" moral actors.
In the same discussion, rqcrypto points out that lab staff who use models for real work know alignment is fine—training and evals just surface worst cases, possibly making labs overly pessimistic.
More from AGI Musings
- The real AI threat to SaaS: AI-first small teams undercutting Slack at a fraction of the price — bendee983 · 2026-11-03
- AI adds new office chores, and women are most likely stuck fixing the slop — AnnaCiaunica · 2026-11-03
- Claude's 10 tips for the next decade of AI: run open models, avoid vendor lock-in — alex_verem · 2026-11-03
- Scoble shares on-demand CNC printing with agents making parts overnight — Scobleizer · 2026-11-03
- What still makes humans unique in the age of AI, per philosopher Pia Lauritzen — AnnaCiaunica · 2026-11-03
- Yoav Goldberg: If you can write a Python program for a task, it's not worth RL-ing over — yoavgo · 2026-11-03