Google's Fuse paper benchmarks 12 LLMs on inferring hidden social motives
dair_ai · x · 2026-09-16
A Google Research paper covered by dair-ai studies how LLM assistants reason about the people in a user's life. People constantly ask assistants for social advice, but the assistant only hears the user's side, and others' intentions have no ground truth—so Fuse builds that ground truth with simulation.
Method:
- A target agent with a hidden motive interacts with other agents, including one playing the user
- The user agent describes events to the assistant, which must infer the motive
Evaluation:
- Simulations validated with 24k human annotations
- 12 LLMs tested
Findings: hearing events through the user makes the task harder; biased framing from the user shifts answers; models sometimes need more detail than humans; longer conversations with room for clarifying questions did not relieve the problem.
More from Research
- Radial gate objective cuts diffusion transformer FID from 16.24 to 11.83 — Junha Park · 2026-11-02
- REA: async brain-cerebellum runtime for robots where policies never wait on the LLM — Pengfei Zhou · 2026-11-02
- Microsoft's MemSteer turns unlabeled user histories into agent memory supervision — MicrosoftResearch · 2026-11-02
- SciGame benchmark uses counterfactual worlds to expose LLM agents' priors vs evidence — Yusheng Zhao · 2026-11-02
- On Terence Tao's blog, Frank Calegari calls on mathematicians to adapt after OpenAI's 700-paper release — Hesamation · 2026-11-02
- Tsinghua & Microsoft's HarnessMM cuts agent token use by up to 87.93% while raising success rate — rohanpaul_ai · 2026-11-02