AI2's TutorMoments eval: GPT-6 models over-help least, detailed prompts beat "tutor well"
allen_ai · x · 2026-10-13
Allen AI built TutorMoments, an eval where experienced teachers marked moments in math tutoring transcripts when tutors should offer support or push students further. Claude Sonnet 5.5 (low effort) was best at letting students work through problems themselves when recommended; with explicit tutoring instructions, GPT-6 Sol (no reasoning) outscored Claude Opus 4.8 overall. GPT-6 models over-help far less than earlier OpenAI models, and all models scored better with detailed prompts than a generic "tutor well" instruction.
Related event: Allen AI's TutorMoments Benchmark Finds LLM Tutors Overdo It for Students(5 posts)→
More from Models
- User feeds Claude Opus GTA frame-by-frame screenshots since it can't watch videos — adonis_singh · 2026-10-13
- Gemini Deep Research on web is really slow today, user reports — gaganghotra_ · 2026-10-13
- Dual RAID10 NVFP4 setup loads GLM 5.3 Flash cold in ~1 minute — TheZachMueller · 2026-10-13
- Opus 5.5 stuns at SVG generation; Meta's Muse Wonder judged 3-6 months behind frontier — scaling01 · 2026-10-13
- Muse vs Opus 5.5: same 3D prompt costs $4.11 vs $99.43, big quality gap — iamaliveix · 2026-10-13
- 179M-Parameter Open Retriever UIBK-Embedding-v1 Tops BEIR Under 1B Models — antoine_chaffin · 2026-10-13