Professors built 'AI-proof' exam questions — GPT-5.4 still scored 83%
alex_verem · x · 2026-11-20
Penn State researchers collected 61 'AI-proof' questions from 37 professors across 11 universities, plus 14 ordinary ones, and ran three frontier models 10x each with/without course notes (4,500 graded answers). GPT-5.4 averaged 83% on questions designed to stop it, Claude Sonnet 4.6 got 82%, Gemini 2.5 Flash 79% — only 4-6 points harder than ordinary questions. Of five professor tricks, only added complexity/misdirection reliably worked (-7 to 10 points); format requirements backfired. The authors urge testing tweaked questions on current AI and warn take-home assessments are losing their value as proof of learning.
More from Models
- Gemini 4 Neon one-shot a Shibuya runner game starring Google's Pegman — rseroter · 2026-11-20
- User tests Opus 5.5 via Claude Pro: after 24h it beats ChatGPT subs on value — mariofilhoml · 2026-11-20
- Google AI declares a border configuration 'mathematically impossible' and proves it — wrongly — conitzer · 2026-11-20
- 13 AI models vs. real policymakers on policy questions: 10 models beat the humans — ChenhaoTan · 2026-11-20
- Frontier models can unmask authorship from a single text, no tools needed — MohitIyyer · 2026-11-20
- Redditor slams Google AI search: six months of updates made it 'cheaper, weaker, gaslighting' — GlitterHeart-Bliss · 2026-11-20