Asking Gemini 10,701 questions: Google Search rewrites ~45% of brand picks
dejanseo · x · 2026-11-18
SEO firm DEJAN ran a large-scale experiment: each of 10,701 brand-recommendation questions was asked twice to Gemini 4 Hydrogen — once from memory, once with Google Search grounding.
Results: on average 45% of brands in grounded answers (3.4 of 7.5) came from search; only 1.8% of answer pairs were identical. Grounded thought summaries were 3.5x longer. Notably, in 21% of grounded summaries the model mentioned a "mental check" or "simulated search", sometimes adding "if this were a live system" — while live searches were actually running. Search also overrode stale memory: of 70 outdated brands named in those passages, only 1 reached the final answer.
Data came from daily AI-visibility tracking across 3,503 distinct prompts, 119 sites and 17 languages (62% English), analyzed purely by counts and pattern matching with no LLM scoring.
More from Models
- Cursor's CursorBench 4.0: Gemini 4 Carbon tops at 60.4%, DeepSeek under $1 per task — mattyp · 2026-11-18
- Salesforce's SOV-Transcribe-1 ASR model trends on Hugging Face — salesforce-sovereign-intelligence · 2026-11-18
- Test: Google's AI Mode default Gemini 4 Hydrogen knows almost nothing about brands without search — dejanseo · 2026-11-18
- OpenAI explains the 6.2 influencer/community vibe gap: demo videos used pre-release models — i_dg23 · 2026-11-18
- Ruff author Charlie Marsh: GPT-6.2 was an enormous step-up for my work — charliermarsh · 2026-11-18
- Claude Code ignored an explicit rule and invented timestamps 4 hours ahead — shymecw · 2026-11-18