MentalHealthBench: open benchmark with mental health experts to evaluate AI responses
thekaransinghal · x · 2026-09-24
Ali Malik's team announced MentalHealthBench, a new open benchmark developed with mental health experts to evaluate how AI models respond in realistic mental health conversations.
Unlike simple Q&A benchmarks, it focuses on real-world conversational scenarios, probing model response quality and safety in sensitive support contexts. The project is open-sourced on GitHub with an explainer thread.
More from Research
- Debate clarifies: is LLM 'model-directedness' just a 'current speaker' representation? — rgblong · 2026-09-24
- IEEE launches ASLI 2027 speech and language intelligence conference in Singapore — shinjiw_at_cmu · 2026-09-24
- Researchers dissect Anthropic emotion concepts paper: separate self/other speaker representations — rgblong · 2026-09-24
- IEEE launches ASLI 2027, first conference on audio, speech and language intelligence, in Singapore — shinjiw_at_cmu · 2026-09-24
- Bioinformatician resurfaces 21-year-old "antedisciplinary science" essay, says a new field is being born — lpachter · 2026-09-24
- Heng Li's minisplice: a 7k-parameter CNN halves minimap2 splice junction error rate — anshulkundaje · 2026-09-24