Same-Named Claude Opus 5 Behaved Like a Different Model Hours Apart in Side-by-Side Test
FishingCharming5604 · reddit · 2026-09-24
A Reddit user asked Claude Opus 5 the same 40 questions twice, hours apart, and found the second run looked things up 60% more often, wrote 50% more, and scored 90 vs 59 on source support. Opus 5.5 launched in between, but two other models barely changed.
The author concludes the likely cause was silent backend changes to settings around the model—thinking effort, available tools—rather than a model swap. Lesson: log every setting (including untouched ones) when benchmarking AI tools, or you may be measuring the settings, not the AI.
More from Models
- ChatGPT Voice gains plugins, ChatGPT Work access, and GPT-6 Astra/Sol/Luna support — romainhuet · 2026-09-24
- GPT-6 Luna beats GPT-5.6 Sol on HealthBench Professional at ~34x lower cost — BorisMPower · 2026-09-24
- Theory: model 'nerfing' may come from mixed heterogeneous inference hardware, not intent — michellechen · 2026-09-24
- "Please Don't Start This with LLMs": Backlash Against Max Prime-Style Model Naming — scaling01 · 2026-09-24
- Limite 1B 'Violetto': tiny open-source model claims competition-math wins over far larger systems — tensorqt · 2026-09-24
- MentalHealthBench: Frontier Models Improving but Gaps Remain in Context-Seeking — thekaransinghal · 2026-09-24