Study on 250k Conversations: How Humans Collaborate with AI
Diyi_Yang · x · 2026-08-27
Drawing on a corpus of 249,834 real-world conversations with Claude provided by Anthropic, this study investigates human-AI collaboration patterns. Key findings include:
- Task Criticality: Users delegate both trivial and consequential, hard-to-reverse tasks to AI. As stakes rise, engagement increases, with conversation length nearly doubling for high-stake tasks.
- Human Agency: Human-led collaboration dominates, but users construct their level of agency through prompting. A key tension is whether AI builds human capability or substitutes for effort.
- Friction & Learning: Friction arises in roughly half of conversations but is often productive. Active teaching is common (67% of conversations) but benefits are unevenly realized.
More from Research
- Interpretability paper incoming: in-depth look at testing explanations on modern LLMs — burny_tech · 2026-10-16
- Better-learning models tend to be more interpretable, argues Nick Cammarata — burny_tech · 2026-10-16
- 'Interp was mostly hopeless before — now it's automatable': Cammarata on interpretability — burny_tech · 2026-10-16
- Cornell and Amazon revisit the 16x pretraining compute gap: diffusion LMs need only 3-8x — LucaAmb · 2026-10-16
- AI trained on the fraction of human intelligence that fit the page — r0ck3t23 · 2026-10-16
- Quotient Recurrent Reasoners hit 97.9% on Sudoku-Extreme with 78% fewer loops — burny_tech · 2026-10-16