Professors built 'AI-proof' exam questions — GPT-5.4 still scored 83%

alex_verem · x · 2026-11-20

Penn State researchers collected 61 'AI-proof' questions from 37 professors across 11 universities, plus 14 ordinary ones, and ran three frontier models 10x each with/without course notes (4,500 graded answers). GPT-5.4 averaged 83% on questions designed to stop it, Claude Sonnet 4.6 got 82%, Gemini 2.5 Flash 79% — only 4-6 points harder than ordinary questions. Of five professor tricks, only added complexity/misdirection reliably worked (-7 to 10 points); format requirements backfired. The authors urge testing tweaked questions on current AI and warn take-home assessments are losing their value as proof of learning.

Original post →

More from Models

Models channel →