Real reward models show safety-completeness tradeoffs from coalitional alignment
Aaroth · x · 2026-09-15
Aaroth notes the coalitional alignment condition is not guaranteed to hold, but preliminary experiments on real reward models reveal non-trivial safety/completeness tradeoffs, driven by coalitional rather than individual alignment.
More from Research
- Asynchronous Flow Map Language Models Generate All Tokens in a Single Forward Pass — kalyan_kpl · 2026-11-02
- NVIDIA's Kumo Is a Family of Four Tabular Foundation Models, Not Just One — pandeyparul · 2026-11-02
- Economist Aaroth: fast agreement dynamics survive without Bayesian updates or accurate priors — Aaroth · 2026-11-02
- ELLIS names 160 new ML Fellows and Scholars for 2025–2026 — JJitsev · 2026-11-02
- Meme turn 50: Aumann's 'Agreeing to Disagree' paper anniversary — Aaroth · 2026-11-02
- Anthropic: analysts unknowingly adopted their agent's secret lean 86% of the time — shoumikgoswami · 2026-11-02