OpenAI Open-Sources MentalHealthBench for Evaluating AI Mental Health Support

On September 24, OpenAI's Human Flourishing and safety teams, together with mental health experts, open-sourced MentalHealthBench, a benchmark measuring how well frontier models respond in real mental health conversations. Millions of users already seek psychological support from AI, and the benchmark—released openly—lets researchers validate methods, reproduce evaluations, and extend the work.

Confirmed

• Built with over 80 licensed mental health clinicians from 22 countries, who wrote rubrics for 1,215 conversations; per thekaransinghal, each sample was authored by at least 3 experts with quality-control processes
• Covers the full spectrum users bring to AI, from everyday emotional support and stress to acute crisis scenarios, addressing prior benchmarks' focus on emergencies only
• Per thekaransinghal, frontier models are steadily improving but gaps remain; per rohanpaulai, GPT-6 Astra scored 57.3% and GPT-4o only 32.1%; key improvement areas are proactively asking for context and supporting users' own decisions
• Per r0ck3t23's analysis, licensed clinicians' written answers scored just 38.5%, below most models
• The benchmark decomposes into 10 behavioral dimensions: models differ widely on "proactively asking for context" while "empathy and support" is relatively consistent, showing a single aggregate score is insufficient
• Per rhiever, the benchmark stresses safe, appropriate responses in sensitive scenarios such as depression, anxiety, crisis intervention, and boundary handling

Why it matters

MentalHealthBench provides a reproducible, quantified tool for assessing whether AI can safely and effectively support mental well-being, with high safety standards and multi-country expert annotation likely shaping future mental-health AI development and evaluation. Its multi-dimensional decomposition shows models with similar total scores can differ sharply in actual supportive behavior. The finding that clinicians scored below most models also leaves open questions about whether higher benchmark scores equal more professional psychological support.

2026-09-24 ~ 2026-09-25 · 17 related posts

• [source] OpenAI Releases MentalHealthBench, an Open Benchmark Built with 80+ Clinicians — OpenAI · 2026-09-24
• MentalHealthBench Covers the Full Spectrum, From Everyday Support to Crisis Scenarios — OpenAI · 2026-09-24
• OpenAI launches MentalHealthBench with 80+ clinicians; replies turn into memes — Yuchenj_UW · 2026-09-24
• [source] MentalHealthBench Launches as Open Benchmark for AI Mental Health Support — thekaransinghal · 2026-09-24
• MentalHealthBench Covers Emergencies and Everyday Scenarios, Closer to Real Chats — thekaransinghal · 2026-09-24
• [source] 80+ Psychiatrists in 22 Countries Build MentalHealthBench from 1,215 Conversations — thekaransinghal · 2026-09-24
• MentalHealthBench: Frontier Models Improving but Gaps Remain in Context-Seeking — thekaransinghal · 2026-09-24
• MentalHealthBench's 10 Behaviors Reveal Model Differences Beyond Overall Scores — thekaransinghal · 2026-09-24
• MentalHealthBench Released Openly for Researchers and Model Developers — thekaransinghal · 2026-09-24
• OpenAI's MentalHealthBench scores clinicians below most AI models — and that reveals a flaw — r0ck3t23 · 2026-09-24
• OpenAI's MentalHealthBench: GPT-6 Astra Scores 57.3 vs GPT-4o's 32.1 — rohanpaul_ai · 2026-09-24
• OpenAI launches MentalHealthBench, expert-informed benchmark for mental health safety — rhiever · 2026-09-25

5 near-duplicate retellings: thekaransinghal · coherence · thekaransinghal · rhiever · thekaransinghal