Toby Ord: Linear ECI Growth Means Exponential Growth in Task-Horizon Odds
tobyordoxford · x · 2026-11-14
牛津研究员 Toby Ord 在与 @ramez、@benjtodd 等人的讨论中解释了 ECI 指标的数学性质：ECI 对每个 benchmark 先做逆 logistic 变换，使 logistic 式增长看起来像线性增长。因此 ECI 的线性增长对应概率上的 logistic 增长，进而对应赔率（odds）的指数式增长。
他还补充说，在这一框架下用「恒定风险率」模型刻画任务时长（horizon length）时，时长与赔率成正比；不过他自认该简化模型可能并不完全准确。这段讨论涉及 AI 能力外推方法的核心争议：如何把 benchmark 成绩换算成对任务复杂度/时长的预测。
Related event: Oxford's Toby Ord: RL Compute Needs 10,000x Boost to Lift Benchmark Scores(17 posts)→
More from Research
- Cambridge mRNA therapy regrows heart muscle after attack, boosts pumping 10% in mice — kimmonismus · 2026-11-14
- Data curation RL is just SWE envs: code pretraining data increasingly consists of Claude-built, less-sloppy repos — willcb · 2026-11-14
- NeurIPS 2025 best paper: diffusion models start memorizing at ~300 steps per image — Mastbubbles · 2026-11-14
- TestPrune: pruning regression tests to unblock AI-driven autonomous bug fixing — burkov · 2026-11-14
- Toddlers learn new verbs only from responsive partners; a call to rerun the test with AI — AnnaCiaunica · 2026-11-14
- Context beats model choice: Databricks benchmark shows 91% vs 58% for data agents — CautiousUse8597 · 2026-11-14