PNAS study finds unrestricted GPT-4 helps practice scores but hurts later learning by 17%
ValerioCapraro · x · 2026-07-23
A PNAS field experiment with nearly 1,000 high-school students found that generative AI can improve performance during practice but hurt later independent learning.
Study setup
Students in math classes were split into three groups:
- textbooks and notes only
- a standard GPT-4 interface
- a safeguarded GPT-4 tutor that gave hints instead of direct answers
They practiced problems first, then took a closed-book, closed-laptop exam on similar questions.
What happened
- During practice, AI helped students look much better:
- standard GPT-4: +48%
- safeguarded tutor: +127%
- But after the AI was removed, students who used standard GPT-4 scored 17% worse than those who never used AI.
- The safeguarded tutor eliminated the negative effect, but did not improve independent exam performance.
Interpretation
The paper argues that many students asked for answers and copied them, which made the practice session look productive while weakening actual learning.
Related event: PNAS Study: Unconstrained GPT-4 Use Hinders Independent Learning(2 posts)→
More from Research
- LightMTP: lightweight latent multi-token prediction boosts LLM planning in pretraining — gdm3000 · 2026-10-27
- EMNLP study: AI literary translation reads fine but varies more, even within one book — tuvllms · 2026-10-27
- Lab monkeys at $30-80K each: a 4-year breeding cycle is the real preclinical drug bottleneck — zebird0 · 2026-10-27
- OpenAI Foundation funds UNC tumor antigen project, hiring senior experimental scientist — iskander · 2026-10-27
- Steering attention heads toward honesty: up moral patienthood, down psychopathy — edelwax · 2026-10-27
- Are transcoders overkill? New paper argues plain neurons suffice for circuit analysis — burny_tech · 2026-10-27