PNAS study finds unrestricted GPT-4 helps practice scores but hurts later learning by 17%
ValerioCapraro · x · 2026-07-23
A PNAS field experiment with nearly 1,000 high-school students found that generative AI can improve performance during practice but hurt later independent learning.
Study setup
Students in math classes were split into three groups:
- textbooks and notes only
- a standard GPT-4 interface
- a safeguarded GPT-4 tutor that gave hints instead of direct answers
They practiced problems first, then took a closed-book, closed-laptop exam on similar questions.
What happened
- During practice, AI helped students look much better:
- standard GPT-4: +48%
- safeguarded tutor: +127%
- But after the AI was removed, students who used standard GPT-4 scored 17% worse than those who never used AI.
- The safeguarded tutor eliminated the negative effect, but did not improve independent exam performance.
Interpretation
The paper argues that many students asked for answers and copied them, which made the practice session look productive while weakening actual learning.
Related event: PNAS Study: Unconstrained GPT-4 Use Hinders Independent Learning(2 posts)→
More from Research
- Kimi K3 may be strong on cyber, but token efficiency keeps it off UK AISIS — teortaxesTex · 2026-07-27
- ARC AGI 3 should have stayed private, with no examples or public dataset — flowersslop · 2026-07-27
- ExploitGym may have only 60–70% solvable tasks, fueling the OpenAI cheating debate — max_paperclips · 2026-07-27
- Noahpinion quotes Chollet: intelligence may hit a hard ceiling — binarybits · 2026-07-27
- Paper argues graph topology can become the core operating system for AI agents — theomitsa · 2026-07-27
- A question probes how multi-agent branching scales against compute budget and model size — iskander · 2026-07-27