Massive Divergence in Recommendations Across Nine Model Tiers
anon1959 · reddit · 2026-07-13
The author pitted multiple tiers of Claude, GPT, Gemini, and Grok against each other on 10 "best dev tool" questions to verify: will different tiers from the same model family give different recommendations?
Test scope includes:
- Claude: Haiku / Sonnet / Opus
- OpenAI: GPT 5.5 / 5.6
- Gemini: Flash / Pro
- Grok: Fast / Expert
Questions cover vector databases, coding assistants, LLM observability, RAG frameworks, GPU clouds, TTS APIs, gateways, agent frameworks, evaluations, and API providers. The results are quite interesting:
- Out of 10 questions, not a single #1 recommendation was unanimously agreed upon by all 9 tiers
- Splits exist even within the same family: Claude 4/10 consistent, Gemini 5/10, GPT 6/10, Grok 7/10
- Clear "self-preference" or anti-preference phenomena emerged: some tiers recommend their own products, while others recommend competitors
- Flagship tiers generally lean towards CoreWeave; lower-priced tiers prefer cheaper options like Lambda and pgvector
The author acknowledges this is just a single sample and could be affected by randomness, but it highlights an often-overlooked point: when people test "what ChatGPT thinks of my product," they might only be interacting with the flagship model, whereas free/lower-tier users face an entirely different recommendation logic.
More from Venture
- Indie newbie goes from first hearing about overseas SEO to first sale in 21 days via new-keyword play — gefei55 · 2026-10-27
- Bedrock Robotics raises $600M as its fully autonomous excavators do paid work on live job sites — Sethwinterroth · 2026-10-27
- Anthropic files S-1: early investor says 10x YoY growth 'unseen in 25 years of venture' — mmurph · 2026-10-27
- Menlo Ventures partner recalls bending VC rules to back pre-launch Anthropic at $4B — mmurph · 2026-10-27
- Founder generates 46 app promo videos in 4 days with Claude, skipping $11,500 in UGC fees — PrajwalTomar_ · 2026-10-27
- How AI rebuilt his client pipeline into a $20K/month web design business — Murky_Explanation_73 · 2026-10-27