Artificial Analysis Index weights are arbitrary and shift wildly over time
HamelHusain · x · 2026-11-12
Hamel Husain showed that the Artificial Analysis Intelligence Index is made up of 9 benchmarks with arbitrary weights — and those weights vary quite a bit over time. charliermarsh mocked it as 'turning a big dial that says index weight on it and constantly looking back at the audience for approval.'
More from Models
- Claude app widget bugs: edits invisible to Claude, mobile remote control broken — petergyang · 2026-11-12
- Allen-Zhu's new paper: Canon layers boost reasoning 2x, even random convolution weights help — teortaxesTex · 2026-11-12
- Ricursive releases open-weight Ripple-65B, claiming US open-weight SOTA on TerminalBench 2.1 — StanfordAILab · 2026-11-12
- A criminally underrated resource comparing tool-calling reliability across models and providers — ricklamers · 2026-11-12
- Datalab fast-mode OCR beats Opus 5.5 and GPT-6.1 Sol at 1/4 to 1/68 the price — VikParuchuri · 2026-11-12
- Claude's world knowledge impresses: model guesses user identity from one prompt — adonis_singh · 2026-11-12