GPT-4 tutoring lifted practice scores 48% but left students 17% worse on exams
tallmetommy · x · 2026-11-12
Replying to Fchollet's thread on conversational AI transforming education, citing the Bastani et al. study on 1,000 high schoolers:
- Plain GPT-4 chat boosted practice scores by 48%
- But on later exams without AI access, those students scored 17% below peers who never used it — answers were outsourced, thinking wasn't learned
- A hint-only tutor mode largely erased the harm
The core argument: most current education AI tools optimize for the wrong thing — getting the answer — when the point of homework is learning to think through the process. AI should help students develop their own thinking rather than outsource it.
More from AGI Musings
- Rutger Bregman flips: the 'crazy' AI forecasters keep being right, and 2027 could end in catastrophe — jammastergirish · 2026-11-12
- repligate floats the idea we may have already lost control 3 months ago — repligate · 2026-11-12
- X user asserts Google will be the ultimate winner of superintelligence — imjustnewatai · 2026-11-12
- Liang Wenfeng's AGI-first bet revisited: why image and video generation are 'detours' — teortaxesTex · 2026-11-12
- Brundage recommends Kevin Roose's new book 'The AGI Chronicles' — Miles_Brundage · 2026-11-12
- AI progress tracker: Anthropic's internal Model 4 runs ~6 weeks ahead of AI-2027 scenario — haider1 · 2026-11-12