GPT-5.6 Sol vs Claude Fable 5 Doodle Test Sparks Hallucination Debate

On 07-12, @goodside used a meaningless doodle created by "drawing blindly on a phone" to test the handwriting reading capabilities of GPT-5.6 Sol and Claude Fable 5. The results were starkly different: Sol consistently fabricated an answer, showing a clear tendency to hallucinate, while Fable 5 directly admitted it couldn't read it and even argued it didn't look like writing at all. This comparison was widely shared in the community as a straightforward case study of how current models differ in "knowing what they don't know."

Test Design

The material itself was a deliberately meaningless input—random scribbles with eyes closed, neither text nor a clear pattern. Therefore, any answer "reading out" content was considered a fabrication. @goodside's initial two examples both set Sol to the "high" effort level, without trying higher levels.

Subsequent Corrections and Controversy

On 07-13, @goodside updated that Sol Pro and Sol non-Pro at higher effort levels could often handle similar examples correctly, sharing ChatGPT links. However, he subsequently ran 3 attempts on 5.6 Sol Pro and still observed hallucinatory behavior with personalization and memory disabled. In response to claims that higher tiers solve the problem, he emphasized that under conditions of a brand-new dialogue with personalization and memory turned off, 5.6 Sol Pro consistently reproduced the hallucination 3 out of 3 times, noting that the previously mentioned third example still persisted. Overall, whether the issue disappears with higher effort/Pro tiers yields inconsistent conclusions across different users and testing conditions.

2026-07-12 ~ 2026-07-13 · 5 related posts

1 near-duplicate retellings: goodside