IIT Madras CeRAI unveils conversational AI eval tool with 7 dimensions, 48 core metrics
ravi_iitm · x · 2026-11-17
At the Industry Conclave on AI Governance 2026, Dr. Sudarsun Santhiappan (WSAI, IIT Madras) ran a hands-on workshop on the Conversational AI Evaluation Tool built by CeRAI, framed by the motto "what cannot be measured cannot be governed."
- Targets: REST APIs, web apps, and WhatsApp bots
- Coverage: 7 evaluation dimensions, 48 core metrics, 120+ sub-metrics, 20K+ curated test cases
- Multilingual: 7 languages including English, Tamil, Hindi, Bengali, Assamese, Sindhi, Bodo
- Outputs: interactive dashboard, detailed reports, per-metric scores
More from Safety
- Skeptic: OpenAI can't formalize 'this AI won't murder you' in Lean — zetalyrae · 2026-11-17
- OpenAI defends evaluator access terms; ex-researcher urges honoring commitments ASAP — j_asminewang · 2026-11-17
- Poll shows 70% of Americans back regulating AI to slow new model development — zetalyrae · 2026-11-17
- Lawmaker questions Commerce: Megaspeed kept buying Nvidia chips after blacklisting — teortaxesTex · 2026-11-17
- Anthropic details four categories of unintended Claude actions, from paywall bypass to URL tricks — JustinLin610 · 2026-11-17
- MIRI launches "Do Not Accept" campaign against AI companies deciding humanity's future — zetalyrae · 2026-11-17