Leaderboard Shifts After Removing SimpleQA
scaling01 · x · 2026-07-19
A screenshot shows a model leaderboard where removing SimpleQA alters the scores and rankings of multiple models. For instance, GPT-5.6 Sol retains the top spot, while models like Claude, Gemini, and Kimi see varying degrees of upward or downward movement. The poster uses this to call out individuals who are "fabricating numbers" to arbitrarily downgrade ECI in order to force a specific conclusion.
Related event: Debate Over SimpleQA's Impact on Model Rankings(3 posts)→
More from Models
- KOL defends B-tier rating for GPT-6.1 Sol, blames OpenAI's botched release naming — Angaisb_ · 2026-10-27
- php2shell exploit benchmark: only 2 of 5 frontier models can port the bug — xeophon · 2026-10-27
- Qwen Code v0.26.2-preview.0 adds session permission rules and experimental shared agent loop — qwen-code-review-bot · 2026-10-27
- GPT-6.1 Terra cuts factual error rate from 22.3% to 4.3%, an ~81% drop — haider1 · 2026-10-27
- GPT-6.1 Terra lands: early users praise speed, value and subagent performance — BenBajarin · 2026-10-27
- Leaked GPT-6.1 Terra benchmarks: cheap but 'boring', still far behind Astra and Claude — Angaisb_ · 2026-10-27