Claude Opus 5 Shows Erratic Behavior in Probability Tests
Benchmark screenshots reveal that Claude Opus 5 exhibits unstable reasoning, repeatedly changing its answer up to 30 times on a single probability question before converging, raising concerns about its reliability.
2026-07-25 ~ 2026-07-25 · 3 related posts
- Episode 1: Opus 5 Coding Paradox: Higher Reasoning Leads to Lower Scores(2026-07-24, 13 posts)
- Episode 2: Claude Opus 5 Shows Erratic Behavior in Probability Tests(2026-07-25, 3 posts)
- Episode 3: GPT-5.6 Beats Claude Opus 5 in Bug Hunt Benchmark(2026-07-25, 3 posts)
- Claude Opus 5 flips between 1/3 and 2/5 before finally settling on an answer — Sauers_ · 2026-07-25
- Claude Opus 5 flips 30 times on a probability problem before settling on one answer — scaling01 · 2026-07-25
- A second Claude Opus 5 screenshot shows the same kind of reasoning instability — scaling01 · 2026-07-25