RCT: ChatGPT temporarily boosts consultants' data science skills, gains vanish without it
EricTopol · x · 2026-10-26
A randomized controlled trial in Nature Human Behaviour (N=487) gave management consultants ChatGPT access and training to tackle data science tasks outside their skill sets.
- Treated workers scored significantly higher than controls on coding, statistics, and prediction tasks — though still below expert data scientists.
- Post-experiment, they showed no advantage answering technical questions without ChatGPT, suggesting no durable knowledge acquisition.
The authors frame generative AI as an "exoskeleton": it enhances capabilities while in use, but the gains depend on continued reliance on the tool.
More from AGI Musings
- NBER: Only 2.9% of US consumers pay for AI; retention as low as 20%, pricing power at risk — avicgoldfarb · 2026-10-26
- AI researchers guessed robots wouldn't outrun humans until 2034 — it already happened — ben_j_todd · 2026-10-26
- AI folks, stop saying "biology" when you mean drug development — KevinKaichuang · 2026-10-26
- ESR on the Bugpocalypse: locking down frontier models only hands the game to Chinese open weights — colinmcnamara · 2026-10-26
- Anthropic: junior tech hires fell from 27% to 9% as agent hours hit 6x employee hours — thione · 2026-10-26
- Anthropic's Jascha: you may have only 1–5 years of intellectual contribution before AGI — GabGarrett · 2026-10-26