GradeBench: best frontier model matches instructors only 57% on real handwritten physics
scaling01 · x · 2026-11-17
- GradeBench is out, built from 6M real student answers across 200+ universities to test AI grading of handwritten, open-ended work.
- The contrast is stark: AI can tackle Millennium Prize Problems, yet the best frontier model agrees with the instructor on only 57% of real handwritten college physics answers.
- Released as an 8-post thread, highlighting AI's weakness on unstructured real-world tasks combining handwriting recognition and subjective scoring.
More from Models
- gemini-4-neon surfaces in Gemini web app code as Google's next unreleased model — imjustnewatai · 2026-11-17
- Pangram is polarizing but the author measured ~1 false positive in thousands of runs — ctjlewis · 2026-11-17
- Kimi K3.1 Pro tops Science's Last Exam among open models at 28.92, trailing only GPT-6.1 Sol and Claude Opus 5 — NielsRogge · 2026-11-17
- Gemini 4 Carbon opens to paying API customers and enterprises, plus 2027 superhuman-coder forecast — thione · 2026-11-17
- Anthropic launches Claude Fable 5.2 and restricted Mythos 5.2, formalized Fermat's proof in 30 hours — thione · 2026-11-17
- Marker OCR upgrade lifts rotated-page pass rate from 81.4% to 87.3% — VikParuchuri · 2026-11-17