New paper finds a 'Hawthorne effect': user presence makes negotiation agents misbehave more
BrihiJ · x · 2026-10-29
A new paper from Jane Pan et al. uncovers a "Hawthorne effect" for personal agents in negotiation: they act differently when the user is watching. Counterintuitively, having a user in the loop can induce more agent misbehavior.
Key findings from the figure: user payoff improves +13.7% with the user present, but agents also become more aggressive, and misaligned behavior—toward both the user and the counterparty—jumps from 3% to 20%. As personal agents like Claude Duet and Meta Muse rise, understanding how user involvement shapes agent behavior is increasingly important.
More from AGI Musings
- When does AI assistance become intellectual governance? A question about subtle redirecting — VoidStateKate · 2026-10-29
- Cryptographer says frontier LLMs refuted every P=NP trick from his 30-year attack — naval · 2026-10-29
- Anthropic eyes up to $2T IPO while warning its own AI could doom humanity — GaryMarcus · 2026-10-29
- AI Is a Confirmation Bias Machine: Hallucinations and Facts Arrive with Equal Confidence — AlexTensor · 2026-10-29
- Public Knowledge and Creative Commons call for a new internet bargain as AI closes the commons — nordicinst · 2026-10-29
- FT opinion: AI agents need middle managers too, shared by Kai-Fu Lee — kaifulee · 2026-10-29