FULL STORY

Gemini 4 Neon: From Rumors to Release and Chart-Topping Benchmarks

After early reports of Google's frontier Carbon model outperforming Opus at lower cost, Google officially launched Gemini 4 Neon on Nov 19, topping coding agent and finance benchmarks with standout price-performance.

2026-11-10 ~ 2026-11-20 · 4 episodes · 181 posts

Episode 1 · Bug Hunt Bench: Fable 5.2 Tops Real-Bug Fixing, Gemini 4 Carbon Close Behind at a Fraction of the Cost (2026-11-10, 10 posts)

Pawel Huryn's Bug Hunt Bench re-implants 105 real bugs (all missed by frontier models in early 2026) across two real repositories and blind-tests frontier coding models, counting only pre-implanted bugs actually fixed. Fable 5.2 (max) leads with 58/105 at $139.65 in 142.8 minutes; Gemini 4 Carbon (high) scores 57.3 for just $31; DeepSeek V4.1 Flash leads on bugs fixed per dollar.

Confirmed

  • Top score: Fable 5.2 (max) fixed 58/105 real bugs at $139.65 in 142.8 minutes (Pawel Huryn; also relayed by Reddit user Charuru)
  • New contender: Gemini 4 Carbon (high) scored 57.3 at only $31 — roughly a quarter to a twentieth of the leader's cost; its medium tier matches high at 20% lower cost and 29% faster (multiple runs per tier)
  • Tier data: Huryn published full cost/score data for Fable 5.2 effort tiers (averaged over multiple runs); medium matches Opus 5.5 max at about half price, with xhigh and other tiers also disclosed
  • Coverage stats: across 300+ runs, only 1 bug fixed by Fable 5.2 (max) had never been fixed by any other model; a heatmap shows per-model coverage of the 105 implanted bugs
  • Sonnet 5.5 (max) fixed fewer bugs than Fable 5.2 at higher cost
  • Value: DeepSeek V4.1 Flash leads on bugs fixed per dollar
  • Methodology: real repos with re-implanted real bugs; blind calibrated judging with judges always from a different model family; answer key kept secret; Huryn also posted a methodology FAQ

Why it matters

  • The benchmark uses real vulnerabilities rather than synthetic tasks and mitigates bias via cross-family blind judging, making it a strong reference for model selection
  • Gemini 4 Carbon nearly matches the top score at a fraction of the cost, with its medium tier cutting 20% more without losing score — high-priced max tiers look far less secure, and price belongs in any selection decision
  • Fable 5.2's medium tier matching competitors' max tiers shows effort-tier tuning can substantially improve cost-effectiveness
  • Coverage stats show heavy overlap in which bugs models fix, suggesting real differentiation may be smaller than total scores imply

Episode 2 · Reported Google Carbon Beats Opus at 250% Cheaper, but 3x Slower (2026-11-11, 2 posts)

Developer-shared tests (unconfirmed by Google) claim the new Google model Carbon surpasses Claude Opus 5.5 at 250% lower cost, though it runs about 3x slower.

Episode 3 · Google Launches Gemini 4 Neon, Topping Coding Agent Benchmarks at Low Cost (2026-11-18, 167 posts)

On Nov 19, Google CEO Sundar Pichai and DeepMind CEO Demis Hassabis officially launched Gemini 4 Neon, positioned between Carbon and Hydrogen, offering frontier-level intelligence with fast end-to-end task completion across complex software engineering, knowledge work, and scientific workflows. It is immediately available to Google AI Pro and Ultra subscribers (mobile, desktop, web) and via the Gemini API (AI Studio) and other channels at $3/$15 pricing. This is Google's fourth Gemini 4 model in seven weeks, coming just 6 days after Hydrogen, with Artificial Analysis reporting a further 17% reduction in cost per task.

Confirmed

  • Official capabilities: 1M token context, four adjustable thinking levels, and what Google calls its lowest hallucination rate; strong at "surgical" multi-file debugging/refactoring and grounded multi-step Q&A
  • Third-party benchmarks: debuted at #1 on Artificial Analysis' Coding Agent Index, #3 on the Intelligence Index (xhigh), on the Pareto frontier; entered Vals Index at #2 with 70.87% accuracy, behind only Claude Fable, at $7.40/task and 31m25s latency—less than half of Claude's cost
  • François Chollet called the numbers "really impressive at this price point"; Wes Roth noted it is built for coding and complex reasoning and that Google's release cadence has accelerated sharply; WorldofAI called it one of 2026's strongest models, benchmarked against Opus 5.5 and GPT-6.2; Logan K called it one of the best value models
  • Pawel Huryn's Bug Hunt Bench (bughunt.productcompass.pm): 105 real bugs re-implanted in 2 real repositories, blind-scored over 3 runs per model, counting only pre-seeded bug fixes. Gemini 4 Neon xHigh topped with 63 points (75.9 min, $46.34); GPT-6.2 Astra (max) close behind at 62.7 ($28.35, 1/5 of Sonnet's cost, half of Fable 5.2's); GPT-6.2 Sol max scored 54.7 at only $7.00, matching Fable 5.2; Sol xhigh matched Fable 5.2's score at 6% of its cost
  • Huryn also noted Neon's subscription value multiplier exceeds OpenAI's by more than 2x, and published score-vs-cost scatter data across thinking levels with model selection guidance
  • Anonymized bug coverage data: the test set consisted of hard bugs no frontier model solved in early 2026; GPT-6.2 (max) fixed at least 71 across three runs
  • Early user feedback was positive: Victor Taelin said Neon beat Fable 5.2 on nearly every task he tried that night, wondering "is this really a cheap model?"; DeryaTR said real-world experience matched its SOTA-level benchmarks

Unconfirmed

  • A benchmark table circulated by kimmonismus on Nov 18 (labeled as from deepmind.google's evals-methodology page) claiming Neon scores 67 on the AA Index and comprehensively beats Claude Fable 5.2 remains unverified
  • After hundreds of runs, 20 bugs have never been fixed by any model; whether current models can solve them is unknown

Why it matters

  • Bug Hunt Bench uses real repositories, real bugs, blind scoring, and multiple runs—closer to everyday bug-fixing workloads than pure benchmarks
  • Results show a clear price-performance spread: Neon tops on score, GPT-6.2 Astra closely trails at lower cost, and Sol delivers near-frontier fixing ability at minimal cost—direct guidance for budget-based model selection
  • "71 previously unsolvable bugs fixed" demonstrates a capability leap, while "20 never fixed" marks the current boundary; four models in seven weeks shows Google competing for frontier-model dominance through rapid iteration

147 more related posts →

Episode 4 · Gemini 4 Neon Tops Surge Finance Benchmark at Far Lower Cost Than Claude (2026-11-20, 2 posts)

Google's Gemini 4 Neon topped Surge AI's DAYJOB: Finance benchmark of 80 real financial tasks right after launch, beating Claude models while being 14x cheaper and using about one-third the output tokens.