Artificial Analysis Intelligence Index v5.0 Sparks Neutrality Backlash as Chinese and Open Models Drop

After Artificial Analysis (AA) released its new Intelligence Index (AAII v5.0), scores for Chinese and open-source models dropped sharply across the board, prompting widespread community criticism of the benchmark's neutrality, with many users seeking alternative evaluation platforms. The dispute currently splits into two camps—"AA is biased" versus "models were overfit"—with no definitive conclusion.

Confirmed

• AA updated its Intelligence Index from v4.3.2 to v5.0. teortaxesTex provided side-by-side screenshots of both versions, showing changes in rankings and scores for models including Claude Opus, with Chinese and open-source models generally losing points in v5.0.
• Reddit user SadTemperature6721 questioned AA's habit of adjusting evaluation metrics around major open-source model releases or company IPOs, arguing its credibility is in doubt and asking for alternative benchmark platforms.
• Reddit user thiswebthisweb complained that AA's leaderboard no longer lists GLM-5.3 Flash (which they described as high-quality and affordable), and likewise asked for alternative benchmark sites.
• X user dealdealignai (as relayed by aiamblichus) accused AA of systematically suppressing scores for open-weight models, and questioned whether "Gemini 4 Argon is really stronger on agentic tasks."

Unconfirmed

• Whether AA actually engaged in subjective favoritism remains unsupported—only user allegations exist, with no internal evidence.
• teortaxesTex offered an alternative "overfitting to a narrow benchmark" explanation: v5.0's changes in difficulty and distribution amplified models' overfitting to the old proxy metrics. This is only their personal analysis.

Why it matters

• Using MiMo V2.6 as an example, teortaxesTex pointed out that the model is narrower and less practical than V4.1F-1017, with weaker RL training trajectories, environment diversity, and quality—yet it scored high because it covered AA V4.x's proxy metrics; after the difficulty increase, its drop was even larger than 1017's. This shows that leaderboard overhauls can either expose benchmark-gaming models or be interpreted as manipulation by the evaluation body. Both the neutrality of third-party leaderboards and the "training for benchmarks" phenomenon deserve ongoing attention.

2026-10-28 ~ 2026-10-29 · 7 related posts

Full story(2 episodes)→

• Episode 1: Artificial Analysis Intelligence Index v5.0 Sparks Neutrality Backlash as Chinese and Open Models Drop(2026-10-28, 7 posts)
• Episode 2: Nathan Lambert on the Benchmarker's Dilemma in AI Evaluation(2026-10-28, 5 posts)

• Accusations fly that Artificial Analysis rankings systematically underrate open-weight models — aiamblichus · 2026-10-28
• Reddit user accuses Artificial Analysis of hiding cost-effective Chinese models, seeks alternatives — thiswebthisweb · 2026-10-28
• Reddit users question Artificial Analysis' metric changes around open-model launches — Sad_Temperature6721 · 2026-10-29
• [source] Artificial Analysis V5 index shifts to harder benchmarks; reconstructed difficulty scale puts GPT-4 near zero — scaling01 · 2026-10-29
• [source] Why Chinese models tank on every Artificial Analysis update: benchmark overfitting, says researcher — teortaxesTex · 2026-10-29
• MiMo V2.6 shows benchmark overfitting: it fell harder than V4.1F once evals got harder — teortaxesTex · 2026-10-29
• [source] AA Intelligence Index v5 accused of downranking open-weight Chinese models via 'private tests' — YasufoKiddo · 2026-10-29