Penn Team Proposes "Coalitional Alignment" as Provable Safety Condition for Auto-Approve

Aaron Roth's team (UPenn) posted "Delegating Authorization to Misaligned Agents: Coalitional Alignment and Safe Control" on arXiv on 09-15, directly challenging the auto-approve features of Codex and Claude Code: these mechanisms delegate gating to a reviewer agent and create a feeling of safety, but their premise is trusting that reviewer—why trust an agent that audits code if you don't trust the agent that wrote it? thomasahle relayed and echoed this critique.

Confirmed

• The model is a three-party framework: the user (principal) and each reviewer agent hold their own utility functions; the driver agent repeatedly proposes actions, and reviewer agents approve or veto by comparing them against a baseline, with the difficulty being reviewer-user utility misalignment.
• Core result: in a single-step decision model they give a clean characterization—reviewers need not exactly share your utility; a weaker "coalitional alignment" condition (k-robust coalitional alignment) suffices for safe delegation.
• The characterization extends to long-horizon MDPs with state-dependent utilities and state-changing actions, even with fully strategic driver and reviewer agents; the key tool is the performance difference lemma, lifting the one-shot Q-value cone characterization to full MDPs, where all Nash equilibria are safe.
• Preliminary experiments with real reward models show coalitional alignment need not hold, revealing non-trivial safety/completeness tradeoffs arising at the coalition level rather than the individual level, offering empirical clues for multi-agent governance.

Why it matters

• Auto-approve is becoming the default experience of mainstream coding agents; this work gives the first provable-safety formal condition for delegated review and shows that current products' sense of safety lacks theoretical grounding.
• Coalitional alignment is a far weaker and more practical condition than per-agent auditing; if adopted, it could provide clear safety bounds and verification standards for agent authorization design.

2026-09-15 ~ 2026-09-15 · 10 related posts

• [source] Do Codex and Claude Code auto-approve features actually keep you safe? A researcher says the trust model is broken — Aaroth · 2026-09-15
• The principal-reviewer-driver model behind the safe auto-approve analysis, explained — Aaroth · 2026-09-15
• When reviewer agents are misaligned: can you still guarantee system safety? — Aaroth · 2026-09-15
• Penn's Aaron Roth gives exact LP-duality condition for when reviewer-agent approval keeps a system provably safe — Aaroth · 2026-09-15
• Aaron Roth extends safe-reviewer characterization from one-shot decisions to full MDPs via performance difference identity — Aaroth · 2026-09-15
• [source] Coalitional alignment: a weaker condition that still guarantees multi-agent safety — Aaroth · 2026-09-15
• Coalitional alignment lifts to MDPs: every Nash equilibrium stays safe for the principal — Aaroth · 2026-09-15
• Real reward models show safety-completeness tradeoffs from coalitional alignment — Aaroth · 2026-09-15
• [source] New arXiv paper: k-robust coalitional alignment for safely delegating review to misaligned agents — Aaroth · 2026-09-15
• If you don't trust the coding agent, why trust the agent reviewing it? — thomasahle · 2026-09-15