Evaluating LLM-Simulated Conversations in Modeling Inconsistent and Uncollaborative Behaviors in Human Social Interaction
Ryo Kamoi, Ameya Godbole, Binglin Zhou, Xiaoxin Lu, Longqi Yang, Rui Zhang, Mengting Wan, Pei Zhou
EMNLP 2026
cs.CL
2026-03-18
On 300 professional dialogues, vanilla GPT-4.1 at 30 turns/call scores 9.5/9.7 vs humans 5.2/5.7; taxonomy prompts overshoot, SFT piles on repetition.
Simulating every participant in a meeting with one family of LLMs is now a standard way to prototype social products, test social-science hypotheses, and guess how people might react to a policy. The usual scoreboard treats higher consistency and higher cooperativeness as better simulation.
Real meetings do not work that way. People misunderstand each other, cut in, dodge questions, and repeat what was just said. Those messy turns are what force clarification, persuasion, and compromise. A simulation where everyone stays on topic and agrees on time will look polished and still be a biased stand-in for human social interaction.
Microsoft, Penn State, and USC introduce CoCoEval. One layer keeps the familiar 1-10 Likert scores for consistency and collaborativeness. The other layer tags each turn for 10 behaviors: five inconsistent (logical contradiction, factual inaccuracy, misunderstanding, redundant information, repetition) and five uncollaborative (persistent disagreement, interruption, off-topic, under-answering, unclear intent). The target is not "fewer is better." The target is frequencies close to humans.
CoCoEvalBench is built from QMSum, NCPC, SIM, and IQ2, covering business, academic, and governmental meetings plus debates. The evaluation set has 300 conversations. Given participant metadata, the previous 30 turns, and a short summary of earlier context, the model must write the next 30 turns. Human continuations are the reference. o4-mini is the judge; its Spearman correlation with five annotators on collaborativeness is 0.586, against 0.441 among humans.
The generators are GPT-4.1, GPT-5.1 Instant, and Claude Opus 4. Three recipes: a vanilla "write like humans" prompt; a taxonomy-guided prompt that lists the 10 behaviors and asks for human-like rates; and supervised fine-tuning of GPT-4.1 on 1,450 human conversations. They also vary how many turns are produced per API call (1, 5, or 30). Claude often hits its output limit at 30 turns, so that cell is missing.
Human continuations score 5.2 on consistency and 5.7 on collaborativeness (95% intervals roughly 4.8-5.6 and 5.3-6.0). All 10 behaviors appear; unclear intent, under-answering, off-topic, and repetition are the more common ones.
Vanilla prompting pushes scores toward the ceiling. GPT-4.1 at 30 turns per call lands at 9.5 / 9.7; even one turn per call is 8.2 / 8.6. GPT-5.1 and Claude sit even higher, with Claude at 9.8 / 9.9 for one-turn generation. The fine-grained plots show almost no interruptions or misunderstandings.
Putting the behavior list in the prompt sends the model the other way. GPT-4.1 at one turn per call falls to 1.3 / 1.3; Claude to 1.5 / 1.4. The instruction asked for human rates. The model treats it as a request to perform every type. At 30 turns per call the same prompt barely moves the needle: GPT-4.1 is back at 9.3 / 9.4.
SFT can match the Likert numbers. GPT-4.1 at one turn per call scores 5.3 / 5.7. The turn-level mix does not match. Repetition and redundancy blow up, including stretches of "Exactly / Yeah that makes sense / That's right." A presence penalty does not fix it.
Likert also misses failures that look numerically close. GPT-5.1 with the taxonomy prompt at one turn per call scores 4.6 / 4.5, near the human 5.2 / 5.7. Manual reading finds the same fact being misunderstood over and over. Turn-level detection sees it; a single conversation score does not.
| Setup | Consistency | Collaborativeness |
| Human continuation | 5.2 | 5.7 |
| GPT-4.1 vanilla, 30 turns/call | 9.5 | 9.7 |
| GPT-4.1 taxonomy, 1 turn/call | 1.3 | 1.3 |
| GPT-4.1 SFT, 1 turn/call | 5.3 | 5.7 |
Anyone using multi-agent dialogue as a stand-in for users or meetings will, under default prompts, get scripts that are too cooperative. Policy reactions, conflict dynamics, and mediation tactics will all look smoother than they should.
CoCoEval is a diagnostic, not a leaderboard. Sounding human and matching human conflict rates are different jobs, and neither prompting nor SFT controls the second one reliably. If you simulate social interaction, count interruptions, misunderstandings, and evasions separately. Do not trust a single 1-10 score.
Public multi-party transcripts lean toward government meetings and debates. Business meetings are scarce, and everything is English. o4-mini agrees weakly with humans on rare inconsistent types such as logical contradiction and factual inaccuracy, and it over-flags under-answering and unclear intent.
The task is a 30-turn continuation, not an open-ended meeting from scratch. Frequency matching is relative to a human continuation that shares the same history, which does not automatically transfer to free-form role play. SFT covers only GPT-4.1. Claude has no 30-turn setting. Open-weight small models are untested. Exact per-behavior means live in bar charts; the paper does not print them as a table.