DeepMind paper: moral argumentation training makes LLMs decisive but no better at moral judgment
rohanpaul_ai · x · 2026-10-30
A new Google DeepMind paper ("Consistency Is Not Validity") finds RL training on moral argumentation makes models more decisive without improving moral judgment.
- Trained Gemma-4 4B with self-judged pairwise tournaments on dilemmas extracted from classic novels (verified via logic solver)
- Answer flips on reordered options dropped from 14.3% to 6.6% on MoralChoice
- But sycophancy barely moved (45.3%→44.0%), and moral fable accuracy fell from 56.5% to 53.4%
- Ablations show similar reordering gains with weaker register shift and higher sycophancy
Key takeaway: consistency is not validity — evaluate consistency and correctness separately when benchmarking judgment tasks.
More from AGI Musings
- Dan Wang: London is already a post-AGI city — 'not functional, but fun' — herbiebradley · 2026-10-30
- Philosopher argues AI could usher in a golden age of mathematics—if we redefine mathematical progress — Dr_Atoosa · 2026-10-30
- A 4-person team ships 77,000 changes/month with AI agents — how devs stay relevant — yoobinray · 2026-10-30
- Ben Horowitz: continual learning is the next big tech revolution and value accrues to its builders — brianryhuang · 2026-10-30
- Anthropic API Key Leaked; Agents Detected and Rotated It Within 15 Minutes — tylerjdunn · 2026-10-30
- Alternate 1865: Steam-Powered AI Agents and Britain's 'Red Tie Act' Output Cap — azmackay · 2026-10-30