Anthropic's capabilities metric decoded: a Bradley-Terry fitted parameter like Epoch's ECI
wfithian · x · 2026-11-06
wfithian explains the capabilities metric in Anthropic's report: it's essentially Anthropic's version of the Epoch Capabilities Index (ECI/AECI).
- It's a fitted parameter in a Bradley-Terry-like model from Item Response Theory: a one-unit ECI increase corresponds to an increase in the log-odds of completing a task — same interpretation as chess Elo
- Theory-laden but not wildly so
- Whether it would have increased linearly absent AI help is unclear (the report may argue this), but empirically models' ECI ratings have been rising linearly for a while
More from Research
- ETH Zurich releases full open-source VLSI course: 12 lectures plus OpenROAD-based exercises — Obijuan_cube · 2026-11-06
- UBC and Sony release Solv3R, turning a looped Transformer into a reusable 3D geometry solver — CSProfKGD · 2026-11-06
- Discovery Loop member launches $DISCODEV open AI materials-discovery loop — arkitus · 2026-11-06
- Northwestern turns ordinary smartwatches into hand and gesture sensors with TransfHAR and electrode-based system — realkaranahuja · 2026-11-06
- Three researchers solo-prompting Bing Chat for four weeks gets roasted as a study — Sauers_ · 2026-11-06
- New attention kernels meet real models: where interpretability overlaps systems research — QuentinAnthon15 · 2026-11-06