Benchmark Health Index: auditing 158 LLM benchmarks with SKYEVAL and Alibaba DAMO

jiqizhixin · x · 2026-11-17

Researchers from SKYEVAL and Alibaba's DAMO Academy propose the Benchmark Health Index (BHI) to audit LLM benchmarks, analyzing 158 benchmarks from reports of 117 models. BHI combines three signals — score discrimination, anti-saturation headroom, and capability-weighted adoption — via leave-one-benchmark-out calibration and CRITIC weighting. Robustness tests show removing 30% of models still leaves rankings with Spearman correlation above 0.97. The tool helps select and revisit evaluation sets as models improve.

Original post →

More from Research

Research channel →