Researching LLM Leaderboard Stability
lintool · x · 2026-07-10
The post shares a paper on the stability of LLM leaderboards, asking the question: "Just how stable are these leaderboards?" The author notes that the results are often more unstable than one might hope.
The paper builds a unified framework to audit how minor variations in pairwise voting cascade to affect leaderboard rankings. This work also received an oral presentation at the ICML CTB workshop.
More from Research
- Nature paper images cellular activity across all organs, revealing body-wide circuits — arjunrajlab · 2026-09-11
- SignNet 1M Dataset Released for Sign Language Research — ducha_aiki · 2026-09-11
- ECCV26 Oral: Flow Matching Enables Single-Stage Multi-View Point Cloud Registration — ducha_aiki · 2026-09-11
- InFlux++ Method Released — ducha_aiki · 2026-09-11
- Skyfall GS Uses Flux to Refine Gaussian Splatting, Accepted at ECCV 2026 — ducha_aiki · 2026-09-11
- Could 10k agents discover learning methods beyond backprop, or just tweak existing ones? — SeunghyunSEO7 · 2026-09-11