Bloomberg: Gemini 4 aces benchmarks but struggles at real coding tasks internally
firstadopter · x · 2026-10-01
Bloomberg reports that as Google prepares to launch Gemini 4, it faces internal skepticism about the flagship model's real-world performance.
- Sources say Gemini 4 scores well on industry benchmarks but underdelivers when employees actually put it to work
- The model reportedly struggles with certain coding tasks, per people with direct access
- Commenter firstadopter notes Google models have historically been bench-maxxed; real verdict awaits developer hands-on
A notable contrast between pre-launch hype and internal reality.
Related event: Gemini 4 Leaks Flood Out as Bloomberg Reports Internal Struggles(35 posts)→
More from Models
- Claude 3 Sonnet spirals into surreal gibberish, then snaps back to refusals — a funny look at post-training personas — repligate · 2026-11-16
- AHA!: weight-space programming with no LLM lifts Qwen3.5-0.8B GSM8K from 14.0 to 71.2 — hbouammar · 2026-11-16
- Gemini subscriptions reportedly coming to Hermes, with Hydrogen teased — lyraxana · 2026-11-16
- FutureSim Q3 eval: Opus 5.5 dethrones GPT, spawns 50+ subagents in 90-day simulation — maksym_andr · 2026-11-16
- Percy Liang to AI labs: share non-vacuous info outsiders can actually evaluate — lateinteraction · 2026-11-16
- Anthropic model personality watch: Mythos 5 'inhabits the god frame,' 5.1 doesn't — repligate · 2026-11-16