Gemini 4 sweeps evals: new SOTA on WANDR and Riemann-Bench, 2-3x cheaper Argon
gabriberton · x · 2026-11-02
A wave of third-party evals confirms Gemini 4's strength:
- Perplexity CEO Aravind Srinivas shared early evals of Gemini 4 Carbon: the strongest model they've seen for long-form research, with a new SoTA of 0.857 on the Wide-And-Deep-Research (WANDR) benchmark at $5.86/task, coming to Perplexity Pro/Max users once token capacity allows.
- Alex Dimakis added Gemini 4 Argon to the AutoResearchExam Pareto curve: 2-3x cheaper than other frontier models for nearly the same ML engineering performance ($92.34/task, hidden-test AUARC 0.594).
- PapersWithCode reports Gemini 4 Carbon scoring 0.8 Pass@1 on Riemann-Bench, ahead of GPT-5.6 Sol (0.74), GPT-6 Astra (0.72) and Claude Opus 5.5 (0.7).
The poster wonders whether the universal praise is a bubble — the screenshots say otherwise.
More from Models
- Developers mourn ChatGPT's unmetered era ending: 'The Meter' kills exploratory flow — doodlestein · 2026-11-02
- Anthropic eats a ~$2B ban loss to protect distillation-driven model moat — oran_ge · 2026-11-02
- Even heavy users can't tell which models power the free tiers of ChatGPT and Claude — iScienceLuvr · 2026-11-02
- Grok to embed invisible watermarks in AI-generated text from Nov 7 to comply with EU AI Act — Polymarket · 2026-11-02
- GPT-6.1 medium in fast mode tested on chess puzzles — imjustnewatai · 2026-11-02
- Google AI Ultra (€219.99/mo) user hits quota bug: locked out with 37% of 5-hour limit left — Specific-Cause-1014 · 2026-11-02