Gemini 4 aces benchmarks but struggles at real coding work, insiders tell Bloomberg
socialwithaayan · x · 2026-10-01
According to a Bloomberg report citing people with direct access to the project, Gemini 4 performs well on widely used benchmarks but falls short when Google employees put it to real work, struggling with certain coding tasks. The post mocks a familiar industry pattern: "every model is great until someone uses it for real work."
Related event: Gemini 4 Leaks Flood Out as Bloomberg Reports Internal Struggles(35 posts)→
More from Models
- Claude 3 Sonnet spirals into surreal gibberish, then snaps back to refusals — a funny look at post-training personas — repligate · 2026-11-16
- AHA!: weight-space programming with no LLM lifts Qwen3.5-0.8B GSM8K from 14.0 to 71.2 — hbouammar · 2026-11-16
- Gemini subscriptions reportedly coming to Hermes, with Hydrogen teased — lyraxana · 2026-11-16
- FutureSim Q3 eval: Opus 5.5 dethrones GPT, spawns 50+ subagents in 90-day simulation — maksym_andr · 2026-11-16
- Percy Liang to AI labs: share non-vacuous info outsiders can actually evaluate — lateinteraction · 2026-11-16
- Anthropic model personality watch: Mythos 5 'inhabits the god frame,' 5.1 doesn't — repligate · 2026-11-16