New arXiv paper: k-robust coalitional alignment for safely delegating review to misaligned agents
Aaroth · x · 2026-09-15
- Aaron Roth and collaborators released "Delegating Authorization to Misaligned Agents: Coalitional Alignment and Safe Control."
- Problem: for long-running agents, requiring human approval on every consequential action makes attention a bottleneck, but delegating review to other AIs reintroduces the alignment problem — reviewers may be misaligned.
- Key result: they characterize a condition weaker than individual alignment that is necessary and sufficient for safety guarantees — k-robust coalitional alignment. A threshold rule tolerating k disapprovals is safe exactly when, after any k reviewers are removed, the principal's utility is a nonnegative combination of the remaining reviewers' utilities plus a nonnegative term.
- The characterization extends to sequential control: in a discounted MDP with an arbitrary proposer, per-state safety is both necessary and sufficient.
- Preliminary empirical evidence shows non-trivial safety/completeness tradeoffs among real reward models, driven by coalitional rather than individual alignment.
More from Safety
- Cryptographers push back: claims ECC is likely to break are not consensus — jedisct1 · 2026-11-02
- UK's 'AI sovereignty' push undermined by its own energy and copyright policy — NinaDSchick · 2026-11-02
- Auditing my MCP credential broker: the worst hole was the config file — Relevant_Mousse_9117 · 2026-11-02
- Cambricon co-founder urges shared US-China AI ethics standards for leading models — pstAsiatech · 2026-11-02
- Vitalik's "post-AI cryptography": pseudonyms are dead, per-action anonymity is in — devanshmehta · 2026-11-02
- 6 prompts to see what ChatGPT quietly knows about you — and wipe it — Aiden_Tech_Ai · 2026-11-02