StudentBench: AI tutoring matches human GRE gains at 918x lower cost
gharik · x · 2026-10-31
A new arXiv paper (2609.28470) by Curtis Northcutt et al. introduces StudentBench, an evaluation suite and platform with 175,000+ student-AI messages, comparing tutoring outcomes across 2,383 participants.
- AI tutoring is statistically equivalent to expert human tutoring for GRE learning gains (p=.015), and the best AI tutor beat the human tutor on average in five of seven GRE domains.
- A second study had expert tutors run 2,028 pairwise rubric evaluations of LLM-generated lesson plans and practice problems, separating AI tutors across lesson planning, problem creation, pedagogy, cost, and engagement.
- Standout finding: one AI tutor matched human gains (p=.044) at 918x lower cost — $0.0052 vs $4.81 per percentage point gained.
More from AGI Musings
- AI pushes circuit lower bound from 3.1n to 3.5n as UCLA-backed challenge aims at P vs NP — soumitrashukla9 · 2026-10-31
- What is work for? Cross-national survey of 8,234 adults on meaning in the AI age — soumitrashukla9 · 2026-10-31
- Which universities win top students may hinge on healthy AI-use boundaries, argues researcher — brianryhuang · 2026-10-31
- Researchers debate whether more inference compute can substitute for training compute — teortaxesTex · 2026-10-31
- Five agents can agree because they made the same mistake, krishnan warns with satire — krishnan · 2026-10-31
- Google's earnings offer clearest look yet at the cruel economics of AI search — dinabass · 2026-10-31