Reddit accuses Artificial Analysis of 're-calibrating' benchmarks to bury open-weight models
Rare_Subject2343 · reddit · 2026-11-10
A Reddit thread accuses AI benchmark aggregator Artificial Analysis of bias against open-weight models: whenever models like DeepSeek or GLM near proprietary leaders, AA allegedly "re-calibrates" its benchmarks, pushing open models down its widely promoted Intelligence Index and making business readers conclude Claude/GPT are the only viable choice.
The poster argues AA ignores open models' real coding strength and suspects undisclosed paid influence; AA's stated rationale is correcting "saturated" benchmarks. The attached chart shows the v5.0.1 index with Claude Opus 4.5, Gemini 4 and GPT-6 on top and open-weight models ranked low. The bias claim is unverified — a methodology controversy, not established fact.
More from Models
- Geo benchmark upset: Claude Fable 5.2 ends Gemini's 18-month reign, tops leaderboard at 49 — Afinetheorem · 2026-11-10
- Claude Fable 5.2 nails the Bach Benchmark with first-ever third-inversion chord — madhavsinghal_ · 2026-11-10
- Anthropic system card chart shows internal model beating public Claude models, joked poster — felpix_ · 2026-11-10
- OpenAI's playbook: counter every Anthropic launch within 1-9 days, timeline shows — haider1 · 2026-11-10
- Mythos 2 Max pricing: 4x cost per CVE for 1% more discoveries — illscience · 2026-11-10
- Browser use: Muse Ember matches GPT-6 Luna at 3 cents per task — WenhuChen · 2026-11-10