Toby Ord: Elo and odds are the same measure on a log scale — ECI works the same way
tobyordoxford · x · 2026-11-14
AI researcher Toby Ord explains that Elo and odds of winning are two representations of the same quantity, with Elo being the logarithm of the odds — analogous to decibels vs. W/m² in sound, or magnitude vs. energy in earthquakes.
He extends the analogy to AI benchmarking: ECI and odds of success on a random benchmark task form the same log-scale pair. In head-to-head matchups, the correspondence is:
- probability of winning ≈ benchmark score (growing logistically)
- odds of winning ≈ odds of task success
- Elo ≈ ECI
The takeaway: logistic growth shows up in probabilities, while log-scale metrics like Elo/ECI better capture linear capability scaling.
Related event: Oxford's Toby Ord: RL Compute Needs 10,000x Boost to Lift Benchmark Scores(17 posts)→
More from Research
- Cambridge mRNA therapy regrows heart muscle after attack, boosts pumping 10% in mice — kimmonismus · 2026-11-14
- Data curation RL is just SWE envs: code pretraining data increasingly consists of Claude-built, less-sloppy repos — willcb · 2026-11-14
- NeurIPS 2025 best paper: diffusion models start memorizing at ~300 steps per image — Mastbubbles · 2026-11-14
- TestPrune: pruning regression tests to unblock AI-driven autonomous bug fixing — burkov · 2026-11-14
- Toddlers learn new verbs only from responsive partners; a call to rerun the test with AI — AnnaCiaunica · 2026-11-14
- Context beats model choice: Databricks benchmark shows 91% vs 58% for data agents — CautiousUse8597 · 2026-11-14