rankECE: a ranking-based measure that beats binned ECE for calibration error
_onionesque · x · 2026-09-14
Surfacing a gem from arXiv slop: Chatterjee & Foygel Barber's "A Ranking Approach for Measuring Calibration". It argues ECE cannot be estimated with guaranteed accuracy assumption-free and proposes rankECE, based on comparing points with neighboring predicted probabilities; theory and experiments show rankECE is a better proxy for ECE than common binned approximations.
More from Research
- Swapping pretraining objective cuts entity-swap false-accepts from 46% to 5% with zero training — Reasonable_Royal_621 · 2026-09-14
- LeanDB: Theoric Labs builds a strongly typed Lean frontend for SQL databases — hargup13 · 2026-09-14
- DeepMind looks back on 15 years of AI game research, partners with EVE Online devs — arnicas · 2026-09-14
- ECCV paper demystifies video reasoning: diffusion steps hide parallel trajectories — ziqi_huang_ · 2026-09-14
- Should arXiv reject an AI-discovered cancer breakthrough that passes clinical trials? — IanArawjo · 2026-09-14
- cESA: judging multiple translations at once cuts human eval time and noise — zouharvi · 2026-09-14