PNAS study finds unrestricted GPT-4 helps practice scores but hurts later learning by 17%
ValerioCapraro · x · 2026-07-23
A PNAS field experiment with nearly 1,000 high-school students found that generative AI can improve performance during practice but hurt later independent learning.
Study setup
Students in math classes were split into three groups:
- textbooks and notes only
- a standard GPT-4 interface
- a safeguarded GPT-4 tutor that gave hints instead of direct answers
They practiced problems first, then took a closed-book, closed-laptop exam on similar questions.
What happened
- During practice, AI helped students look much better:
- standard GPT-4: +48%
- safeguarded tutor: +127%
- But after the AI was removed, students who used standard GPT-4 scored 17% worse than those who never used AI.
- The safeguarded tutor eliminated the negative effect, but did not improve independent exam performance.
Interpretation
The paper argues that many students asked for answers and copied them, which made the practice session look productive while weakening actual learning.
Related event: PNAS Study: Unconstrained GPT-4 Use Hinders Independent Learning(2 posts)→
More from Research
- Causal-only attention for non-generative tasks is wasteful, argues HF engineer — antoine_chaffin · 2026-09-11
- Catholic University of Chile researcher: scaling AI feedback is key to sustainable medical education — julianvarascom · 2026-09-11
- Nature paper images cellular activity across all organs, revealing body-wide circuits — arjunrajlab · 2026-09-11
- SignNet 1M Dataset Released for Sign Language Research — ducha_aiki · 2026-09-11
- ECCV26 Oral: Flow Matching Enables Single-Stage Multi-View Point Cloud Registration — ducha_aiki · 2026-09-11
- InFlux++ Method Released — ducha_aiki · 2026-09-11