scaling01: If Grok and Gemini match frontier models, they're probably benchmaxxed
scaling01 · x · 2026-10-18
X user scaling01 offers a rule of thumb: if Grok and Gemini models score as well as frontier OpenAI and Anthropic models, it's likely benchmaxxing. Responding to people reposting DeepSeek's DeepSWE results, he urges waiting for third-party private benchmarks before drawing conclusions.
More from Models
- UGI leaderboard: Gemini 4 Argon writes best at low thinking, least censored at high — Eissa_Cozorav · 2026-10-18
- Claude cracks a 25-year-old NSA cryptography mystery with $36,864 bounty in under 30 minutes — jradoff · 2026-10-18
- Three Years of AI Progress: 50-Msg GPT-4 Cap to 1M Context, 39% to 96% GPQA — alliekmiller · 2026-10-18
- DeepSWE benchmark skeptics told to wait — Datacurve points to fully public repo — scaling01 · 2026-10-18
- Redditors Still Run 'Obsolete' Local Models Like Mistral 7B Finetunes — Cerebral_Zero · 2026-10-18
- SubQ Snap launches: 384K context, sub-second 100K-token decisions at $0.05/M tokens — eyishazyer · 2026-10-18