AI2 details TutorMoments: teacher-annotated eval measuring whether AI tutors over-help
allen_ai · x · 2026-10-13
Allen AI explains how TutorMoments was built: experienced teachers annotated moments in math tutoring transcripts where a tutor should offer support or ask more of the student. Frontier models including GPT-6 Astra and Claude Opus 5.5 continued those conversations and were scored against the teachers' judgments. Models generally scored better with detailed prompts than when told only to "tutor well."
Related event: Allen AI's TutorMoments Benchmark Finds LLM Tutors Overdo It for Students(5 posts)→
More from Research
- Try AA-3D Voting yourself: blind-test AI-built 3D scenes — ArtificialAnlys · 2026-10-13
- Announcing AA-3D: Artificial Analysis benchmarks AI models building interactive 3D scenes — ArtificialAnlys · 2026-10-13
- Same models top almost every AA-3D domain, Artificial Analysis finds — ArtificialAnlys · 2026-10-13
- One-parameter model scores 100% on ARC-AGI-2 — by deliberately cheating with chaos theory — fkasummer · 2026-10-13
- Desk rejected by TMLR in 2 days with no explanation, laments undergrad author — Audaticreddit · 2026-10-13
- DUET paper co-evolves agent and LLM grader, hits 96.5% on Finance Agent benchmark — dair_ai · 2026-10-13