Gemini 4 Scores Badly on cua-bench, Fueling Benchmaxxing Concerns
burny_tech · x · 2026-10-01
cua-bench creator spiceylemonade pushed back on Gemini 4's strong official scores: on their computer-use benchmark, Gemini 4 scores badly ("perhaps antigravity needs some work?"). The benchmark requires playing Minecraft, SUPERHOT, and unrevealed held-out games to prevent hillclimbing, though it's speed-bottlenecked at the further end.
Context: a cited report claims Gemini 4 performs well on industry benchmarks but underperforms when Google employees actually use it for work, reigniting "benchmaxxing" criticism. The gap between official scores and real-world agent performance is becoming a community flashpoint.
Related event: Gemini 4 Leaks Flood Out as Bloomberg Reports Internal Struggles(35 posts)→
More from Models
- Claude 3 Sonnet spirals into surreal gibberish, then snaps back to refusals — a funny look at post-training personas — repligate · 2026-11-16
- AHA!: weight-space programming with no LLM lifts Qwen3.5-0.8B GSM8K from 14.0 to 71.2 — hbouammar · 2026-11-16
- Gemini subscriptions reportedly coming to Hermes, with Hydrogen teased — lyraxana · 2026-11-16
- FutureSim Q3 eval: Opus 5.5 dethrones GPT, spawns 50+ subagents in 90-day simulation — maksym_andr · 2026-11-16
- Percy Liang to AI labs: share non-vacuous info outsiders can actually evaluate — lateinteraction · 2026-11-16
- Anthropic model personality watch: Mythos 5 'inhabits the god frame,' 5.1 doesn't — repligate · 2026-11-16