Massive Divergence in Recommendations Across Nine Model Tiers
anon1959 · reddit · 2026-07-13
The author pitted multiple tiers of Claude, GPT, Gemini, and Grok against each other on 10 "best dev tool" questions to verify: will different tiers from the same model family give different recommendations?
Test scope includes:
- Claude: Haiku / Sonnet / Opus
- OpenAI: GPT 5.5 / 5.6
- Gemini: Flash / Pro
- Grok: Fast / Expert
Questions cover vector databases, coding assistants, LLM observability, RAG frameworks, GPU clouds, TTS APIs, gateways, agent frameworks, evaluations, and API providers. The results are quite interesting:
- Out of 10 questions, not a single #1 recommendation was unanimously agreed upon by all 9 tiers
- Splits exist even within the same family: Claude 4/10 consistent, Gemini 5/10, GPT 6/10, Grok 7/10
- Clear "self-preference" or anti-preference phenomena emerged: some tiers recommend their own products, while others recommend competitors
- Flagship tiers generally lean towards CoreWeave; lower-priced tiers prefer cheaper options like Lambda and pgvector
The author acknowledges this is just a single sample and could be affected by randomness, but it highlights an often-overlooked point: when people test "what ChatGPT thinks of my product," they might only be interacting with the flagship model, whereas free/lower-tier users face an entirely different recommendation logic.
More from Venture
- Steal this idea: prompt-to-hardware where agents assemble custom devices — paraschopra · 2026-09-11
- Marc Loou gives away free book on making $3M from 36 startups, 90% of them failures — marclou · 2026-09-11
- Trucking brokerage acquired for its AI platform and 8,000-carrier network, not revenue — MatthewChang · 2026-09-11
- Investor argues Palantir-Nvidia partnership should slash Anthropic's IPO valuation — pdamodaran · 2026-09-11
- Moonshot's annualized revenue jumped from $300M to $1B in two months after Kimi K3 — Hesamation · 2026-09-11
- Mid-market companies' AI SEO bottleneck is ops execution, not strategy, says SEO practitioner — gaganghotra_ · 2026-09-11