MLCommons Launches Candela, a Benchmarking-as-a-Service for AI Safety Risk
TheKanter · x · 2026-10-30
MLCommons — an industry body of 125+ members including Microsoft, Google, NVIDIA, OpenAI and Anthropic — launched Candela, benchmarking-as-a-service for AI risk and reliability.
- Core problem: model behavior shifts with every configuration, and there's been no common, believable way to test safety
- Runs consensus-built AILuminate benchmarks on demand, on either a model as released or a system as deployed, producing official, versioned scores graded on a separate prompt set
- Launches with Safety 1.2.1 (12 hazard categories, from violent crime to specialized financial/health/legal advice) and Jailbreak 1.1 (11 categories using its jailbreak taxonomy)
The pitch: AI risk is a normal risk — measurable, and therefore manageable, like the insurance industry's premise.
More from Safety
- House Dems push resolution opposing taxpayer bailouts for failing AI companies — michalmalewicz · 2026-10-30
- Crypto dev eats his words on public Wi-Fi safety after eMudhra TLS surprise — matthew_d_green · 2026-10-30
- Alleged massive Indian KYC data breach mocks CERT-In's 'secure BFSI' cybersecurity drill brag — kingslyj · 2026-10-30
- Alternate 1865: Steam-Powered AI Agents and Britain's 'Red Tie Act' Output Cap — azmackay · 2026-10-30
- Canada takes an equity stake in Cohere to build sovereign AI at home — cohere · 2026-10-30
- Poll: Over 70% of swing-state voters back AI safety measures even at cost to US competitiveness — Miles_Brundage · 2026-10-30